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Abstract 



Evaluation of the Student Engagement Process in a Criminal Justice Program at a 
Technical College. Totzke, Martin W., 2007: Applied Dissertation, Nova Southeastern 
University, Fischler School of Education and Human Services. Learning and Perception/ 
Experiential Learning/Problem Based Leaming/Two Year Colleges/ Andragogy 

This applied dissertation was an evaluation of the student engagement practices offered to 
students in a criminal justice program at a Midwestern technical college. The problem 
was that an evaluation of the practices had not been conducted to provide data to indicate 
whether the current practices were effective. 

The researcher developed an evaluation tool to assess existing student engagement 
activities employed within the criminal justice department at the college. The expected 
criteria of the currently used activities were compared to expected criteria of best 
practices established for a student engagement process at a community college for 
vocational education practices and programs. The expected outcomes (criteria) of best 
practices for a student engagement process at the college criminal justice program were 
developed from the following resources: (a) a review of literature, (b) a survey 
administered to students currently enrolled in the criminal justice program at the college, 
and (c) a survey administered to faculty members of the criminal justice program. 

The evaluation of student engagement practices currently employed at the college 
identified gaps between expected outcomes of the current practices and expected 
outcomes for student engagement best practices model for a criminal justice applied 
associate degree program. 
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Chapter 1: Introduction 

Introduction 

Student engagement is an academic process with a protracted history. During the 
past 3 decades, research and writing focused on student engagement as an important 
aspect affecting the quality of student learning. This applied dissertation project was 
conducted at a Midwestern technical college. Hereafter, the term college refers to the 
subject Midwestern technical college. 

The Wisconsin Technical College System 

The Wisconsin Technical College System (WTCS) was originally established in 
1907 by legislation permitting cities to offer trade school opportunities for youths age 16 
or older. The offerings were to be part of the public school system. This legislation 
mandated communities with a population greater than 5,000 to establish industrial 
education boards that were authorized to levy taxes to support this endeavor (WTCS, 
2006a). 

In 1917, the focus changed from industrial education to vocational education as a 
result of federal legislation. In 1937, the WTCS expanded to offer college transfer credits 
due to a change in the compulsory school attendance requirements and an increased 
demand for adult education. The 1963 Federal Vocational Education Act helped 
transform the WTCS once again. In 1965, state legislature required the establishment of a 
statewide vocational, technical, and adult education system (WTCS, 2006a). 

The state legislature passed enabling legislation establishing the State Technical 
College System. The WTCS, of which the college remains a member, is composed of 16 
technical colleges throughout the state. Each institution serves a geographically defined 
area. Each college is directed to serve the needs and requirements of its individual labor 
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market. The WTCS board of directors establishes the vision, policies, and standards for 
the entire system. Each district appoints a board responsible for focusing the broader 
statewide vision within the individual district. 

Mission and Vision 

In order to address the academic and learning needs of students and the functions 
that the college system provides to meet those needs, the WTCS was assigned a mission 
set forth by the state legislature in enabling legislation. The legislature recognized the 
importance of having a skilled and educated workforce. The technical college system was 
to provide occupational skills training that stressed job training and retraining in order to 
meet the needs of dynamic, ever-evolving employment environment. The technical 
college system was also tasked to provide education and training opportunities to all 
members of the public. 

In addition to the mission, the WCTS (2006b) further addressed the provision of 

meeting the academic and learning needs of students in the following vision statement: 

The Wisconsin Technical College System is the premier provider of technical 
education. We develop individuals who apply knowledge and skills to enhance 
quality of life and boost economic vitality. We are committed to extending learning 
beyond the classroom and throughout life. 

To meet each student’s educational needs, we: 

Deliver high quality instruction and services that are responsive, flexible 
and accessible. 

Join talent and technology to make learning generously available and 
imaginatively delivered. 

Commit to high standards and accountability. 

Create strategic alliances that expand students’ learning opportunities. 

Respect each other’s dignity, embrace diversity and offer opportunities for 
growth, (f 1) 

The Midwestern Technical College 

In addition to the WTCS vision and mission, each college is mandated by WTCS 
to maintain a mission and vision statement. The mandate requires that individual colleges 
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develop mission and vision statements mirroring the mission and vision statement of the 

WTCS. The mission is documented on the college Web site. The mission statement of the 

subject college is to “help individuals reach their potential by providing cost-effective 

education and training which meet their objectives for employment, continuing higher 

education, and personal enrichment.” This statement mirrors the legislative intent and 

direction to prepare residents for participation in the state’s work force. 

The vision of the college is published on the college’s Web site: 

The Strategic Planning Process has led us to a shared vision to fulfill our mission. 
Four strategic directions focus on assuring excellence in all college endeavors, 
maximizing the collective value of those endeavors in the eyes of the college’s 
constituency, reaching more people with programs and services, and 
strengthening the lifelong connection between the college and its students and 
graduates. 

Protect and enhance the performance reputation of the college by seeking 
continuous improvement in all areas. 

Strengthen the college’s value to its constituents. 

Reach more people with programs and services. 

Strengthen lifelong ties to our students and graduates. 

The college serves five counties with an aggregate population of 454,698 
residents (U.S. Census Bureau, 2000). From this population, the college draws more than 
50,000 persons to attend classes. The college offers 75 applied associate degrees and 
technical certificate programs. The college also offers a variety of continuing education 
courses and certificate programs. The college employs 333 full- and part-time instructors, 
as well as 798 adjunct faculty members. In addition, 405 professional and support staff 
contribute to the programs. 

The criminal justice division is the largest academic area of the college. The 
division offers associate degrees in criminal justice and in security loss prevention and 
also offers three technical certificates in the areas of cyber-crime investigations, criminal 
justice leadership, and private investigation. The division also operates three entry-level 
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academies, leadership and management courses, and a plethora of specialized law 
enforcement trainings. The division employs six full-time and 42 adjunct faculty. 

The class of 2005 included 94 criminal justice and 5 security loss prevention 
graduates. The 2006 Graduate Employment Research Report documented that 90% of 
criminal justice and 80% of security loss prevention students responding to the survey 
were employed within 6 months of graduation. The response rates for the graduate 
employment research survey were 76% criminal justice and 100% security loss 
prevention. 

Professional continuing education course offerings occur throughout the calendar 
year. In addition, the division includes an active contract-training component that 
generates revenues in excess of $12 million annually. Contract training expands upon 
mandated entry-level training and education by providing specialized programs and 
projects developed to specific criteria for participants residing beyond the geographically 
defined district boundaries of the college. The scope of contract training, staffed by an 
additional cadre of subject matter experts, is national. Contract training partners include 
The Office of National Missing and Exploited Children, The Office of Juvenile Justice 
and Delinquency Prevention, the National Crime Prevention Council, and the State 
Department of Justice. 

Nature of the Problem 

The criminal justice department enjoys an excellent reputation throughout the 
state. The college’s 2006 Graduate Employment Research Report demonstrates the 
strength of this reputation. The report shows a 90% employment rate 6 months after 
graduation. However, this reputation masks the problem to be studied. The problem is 
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that criminal justice students fell below the 2004 Community College Survey of Student 
Engagement mean in four of the five engagement benchmarks. 

The college participated in the 2004 Community College Survey of Student 
Engagement (CCSSE). The survey is administered through the Community College 
Leadership Program of the University of Texas. The survey was administered to 532 
full-time and 228 part-time students. The criminal justice division was disproportionately 
represented in the sample of survey respondents with 283 criminal justice students 
completing the survey. 

The number of criminal justice students surveyed was linked to the requirements 
associated with the League for Innovation and College and Career Transition Initiative 
(CCTI), in which the college was a partner. The league and CCTI funded the 
administration of the CCSSE with a grant, allowing the college to participate fully in the 
study. Neither the college accreditation committee nor the high school transition 
department utilized the data collected from the CCSSE. 

The director of CCTI reported that a follow-up to the 2004 CCSSE was 
conducted. The CCTI grant did not fund the follow-up effort. Grant funding was essential 
to the college’s participation in the follow-up, and due to the unavailability of funding, no 
follow-up occurred at the college. 

The CCSSE was a relatively new survey tool when the first survey was 
administered at the college. The director of CCTI chose the survey to get a good handle 
on community college student engagement and to identify as many characteristics as they 
could. The survey covered five benchmarks of student engagement: (a) active and 
collaborative learning, (b) student effort, (c) academic challenge, (d) student-faculty 
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interaction, and (e) support for learners (Community College Leadership Program, 2004). 

Student engagement focuses on the amount of effort students invest in their 
learning. Active and collaborative learning relates to how actively involved students are 
in their learning. The academic challenge benchmark inquires into the complexity of the 
cognitive work and course evaluation standards. Academic challenge examines whether 
or not students were afforded the opportunity to solve problems with peers and challenge 
course content. How the faculty engages students was also surveyed using the student- 
faculty interaction benchmark, which looked at how involved instructors were with 
students. The final benchmark focused on support for learners. The survey examined 
college services related to financial aid, advising, and career development, as such 
services may affect learning and retention. In the report Engagement by Design: 2004 
(Community College Leadership Program, 2004), it was noted that such services “may 
affect learning and retention” (p. 12). 

A 2004 report of the subject college discussed benchmark scores and described 

the determination of the mean as follows: 

Every college has a score for each benchmark. These benchmark scores were 
computed by averaging the scores on survey items that comprise that benchmark. 
Benchmark scores are standardized so that the mean— the average of all 
participating students-always is 50 and the standard deviation is 24. 

In an unpublished institutional report on the 2004 CCSSE, the director of institutional 

research reported that the college was below the mean in the remaining four benchmarks. 

The results indicated that the below-mean benchmarks were statistically significant. The 

mean for the benchmarks was determined always to be 50%. The CCSSE established that 

the college was above the mean, 51.8%, in the active and collaborative learning 

benchmark. Specific college results in comparison to other participating institutions are 
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presented in Table 1. 
Table 1 



Community College Survey of Student Engagement 2004 Benchmark Mean Scores 
for All Students at the Subject Midwestern Technical College Compared to Similarly 
Classified Colleges Participating in the 2004 Survey 



Benchmark 


Subject college 


Similar colleges 


Active and collaborative learning 


51.8 


49.0 


Student effort 


47.4 


50.2 


Academic challenge 


46.0 


49.6 


Student faculty interaction 


48.8 


49.2 


Support of learning 


45.0 


49.7 



Note. Benchmark scores are standardized so that the mean is always 50. 

The chairperson of the criminal justice department and the dean of the criminal 
justice division agreed that the survey indicated there was room for improvement. A list 
of the survey items used in the 2004 CCSSE is included as Appendix A. 

Discussions regarding student preparedness among members of the criminal 
justice division resulted in the formation of a committee. The student-involvement 
initiative committee was designated to examine the issues of students’ lack of class 
preparation and participation. The committee was challenged to identify methods to 
increase student involvement. Included in the charge was the identification of existing 
faculty practices designed to increase student involvement. The committee was also 
requested by the division dean to submit recommendations of practices to improve the 
student-involvement process for department and division consideration. Committee 
membership included the department chair and two adjunct instructors. The committee 
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took no action. 

Background and Significance of the Problem 

The issue of student preparation for college is important not only for 4-year 
institutions but especially so for 2-year colleges. Astin (1985) commented that, in 
“virtually every longitudinal study of student development is that the chances of dropping 
out are substantially greater at a two-year college than at a four-year college” (p. 146). 
Accreditation 

The college is accredited through the North Central Association of Colleges and 
Schools. The college participates in the Academic Quality Improvement Program (AQIP) 
for reaccreditation. The college was admitted to AQIP on October 3, 2001. The next 
reaccreditation process will be conducted during the 2008-2009 academic year (Higher 
Learning Commission, 2002). The Higher Learning Commission described the AQIP 
process: 

The Academic Quality Improvement Program infuses the principles and benefits 
of continuous improvement into the culture of colleges and universities by 
providing an alternative process through which an already-accredited institution 
can maintain its accreditation from the Higher Learning Commission. With AQIP, 
an institution demonstrates it meets accreditation standards and expectations 
through sequences of events that align with those ongoing activities that 
characterize organizations striving to improve their performance, (p. 1) 

The college communicated the AQIP accreditation process to the staff and public through 

an Internet posting (see Appendix B). 

The AQIP process examines several areas of the college. One such area is how 
the college helps students learn. AQIP, within the scope of helping students to learn, 
inquires how the college aligns cocurricular activities with course-curriculum objectives 



(see Appendix C). 
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Student Engagement 

The term student engagement refers to the amount of effort students put into their 
studies. Difficulties were noted through personal discussions with criminal justice 
instructors as well as criminal justice administrators at the other state technical college 
criminal justice programs, in getting students to engage subject matter. The consensus is 
that difficulty exists with getting students to read assigned materials or actively 
participate in some courses. Through these discussions, the question arises: Are the 
students ill prepared for the rigors associated with a vocational college, or are they simply 
apathetic? McClenney (2004) commented that only 42% of U.S. high school graduates 
were ready to attend college. This observation is one explanation for the deficiencies in 
academic performance of students. The lack of preparation may exacerbate the 
engagement issue and be a contributor to first-term student retention issues. 

Factors That Affect Student Engagement 

Several factors affect student engagement. Such factors include student 
background, faculty involvement, institutional support of learners, and academic 
challenges. 

Student background. Student background encompasses three influences affecting 
student engagement. These influences are familial and generational, lack of residential 
facilities, and student socioeconomic conditions. 

McClenney (2004), Pike and Kuh (2005), and Warburton, Bugarin, and Nunez 
(2001) all observed that many high school students were simply ill prepared for the rigors 
of college-level study. These observations suggested that many high school students lack 
the academic engagement skills to prepare them for college. 

During staff meetings, the division faculty noted that the college is a commuter 
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institution with a diverse mix of traditional, nontraditional, and displaced students. The 
college offers no residential living facilities for students; therefore, students reside off 
campus, commuting from personal residences to the college. This absence of dormitory 
facilities, along with being located in a metropolitan area, classifies the college as an 
urban commuter college. The lack of dormitory facilities reduces opportunities for 
students to congregate in common areas and debate the essence of their courses. 

The 2004 CCSSE survey suggests that students attending the college are generally 
middle class and maintain employment to finance their education and to provide the 
essentials for themselves and often for their families. The demands placed on student 
time related to employment and familial obligations reduce the time available for 
studying. 

The college offers a variety of cocurricular activities to enhance opportunities for 
student social and intellectual engagement through student life. These opportunities are in 
the form of student clubs, associations, and sporting activities. 

The importance of cocurricular activities is identified in the college’s 
reaccreditation process. One reaccreditation element is helping students to learn (see 
Appendix C, Process Question 1P10). 

An Arizona State University (ASU; n.d.a) Web page identified benefits for both 
students and faculty who engage in external class discussions. Such benefits were 
recognized in the works of several researchers (Jaasma & Koper, 2001; Kuh & Hu, 2001; 
Plecha, 2002). The first benefit that students exhibit is an increased confidence in their 
scholastic abilities. Through informal interaction, instructors gain the opportunity to 
praise the demonstrated effort of a student. Second, improvement in academic 
performance was noted. Students who are struggling with a concept are given the 
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opportunity to discuss the idea with the professor. The conversation may provide the 
instructor with an opportunity to communicate on a level the students more readily 
understand. Third, these students demonstrate greater effort in academic activities that 
they perceive as purposeful. Students gain important insight from the professor as to why 
an assignment or activity was given out. Fourth, students who engage in cocurricular 
discussions with faculty demonstrate increased persistence rates. Educators gain the 
opportunity to reinforce students’ ability to be successful in the course. Fifth, this group 
of students is more likely to desire and to pursue graduate-level studies. Instructors 
acquire the opportunity to reinforce that students possess the intellect to successfully 
study at the graduate level. Sixth, the students integrate into the college environment, 
both socially and academically, better than students who do not communicate with 
faculty. Faculty obtain an opportunity to direct students toward activities the students 
may find rewarding, both collegiate and community based. Such a relationship is more 
individualized than the classroom would allow. 

The ASU (n.d.b) Web page also listed a series of tips for students to assist them to 
improve faculty- student interaction. For a variety of reasons, instructors frequently 
intimidate students. The tips provide suggestions to assist students in overcoming this 
sense of intimidation, allowing enhanced student-faculty engagement and substantive 
interaction (see Appendix D). 

Faculty involvement. At the college, adjunct instructors teach a significant number 
of course offerings. Adjunct instructors outnumber the full-time faculty college wide by a 
ratio of 2.3:1. The criminal justice division ratio is nearly three times higher at 7:1. These 
instructors are hired for their specialized vocational knowledge, skills, and abilities. 
Although they are excellent instructors, the use of adjuncts diminishes opportunities for 
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student engagement. Adjunct instructors are not required to maintain office or advising 
hours. Such times are specifically scheduled for a student to meet with a faculty academic 
advisor or an instructor to discuss academic, course, and career issues. The scheduling 
aspect of these hours advertises times when a student may justifiably assume that the 
instructor will be available for consultation. 

Adjunct instructors are frequently assigned evening course offerings, which 
complicates student-instructor interaction. Students and adjunct instructors often arrive 
directly from work and just in time for the start of the class period. Both the students and 
the instructor are fatigued after class, departing with limited, if any, interest in interaction 
after class. 

Communication and outside-of-class interaction produce benefits for instructors 
as well as students. An ASU (n.d.a) Web page identified three benefits for faculty 
members who engage in external class discussions with students. Those benefits are 
related to student-faculty interaction. First, students submit higher ratings on course 
evaluations for professors who interact with them. The interaction communicates that the 
faculty member cares about the student and that any statements made are more than 
hollow rhetoric. Second, teachers who interact with students receive more nominations 
and recognition for their efforts. Once again, students will recognize instructors who 
relate with them as outstanding educators. Third, students indicate that classes are 
inspiring when taught by instructors who relate well with students. 

The ASU (n.d.b) Web page also listed a series of tips for faculty to improve 
student-faculty interaction. For a variety of reasons, instructors frequently intimidate 
students. The tips provide suggestions to assist faculty in overcoming this sense of 
intimidation on the part of the student, allowing for enhanced student-faculty engagement 
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and substantive interaction (see Appendix E). 

Support for learners. The college provides three programs designed to assist 
students with academic and social issues. The college maintains a peer-advising program 
that utilizes students as advisors. The premise associated with the peer-advising program 
is that students may be more at ease asking questions of another student. 

Student success often requires that the student receive special assistance. The 
college offers special educational services for students requiring individual 
accommodations. Tutoring, special testing conditions, or disability accommodations are 
available to students upon request and are offered through an educational support services 
office. 

The college also offers remedial education for students to improve general 
education skills. The Goal-Oriented Adult Learning program (GOAL) allows students the 
opportunity to improve math, English, and reading skills prior to enrolling in specific 
courses. This program, while improving academic skills, contributes to student 
confidence in intellectual abilities, thereby enhancing the possibility that students will 
engage other students and faculty. 

Active and collaborative learning. Student success is frequently predicated on the 
level of student participation in the learning process. Learning is an active process that 
does not occur without effort. Student participation in classroom-related activities as well 
as collaboration with faculty and peers are important. Astin (1996) reported that peer 
groups exert the strongest influence on the cognitive and affective development of 
peer-group members and, as such, cannot be discounted. Active and collaborative 
learning may occur with learning communities or peer-group interactions focused on 



course content. 
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Purpose of the Project 

The purpose of this study was to evaluate the engagement process of criminal 
justice majors at the college. A recommendation will be forwarded to college 
administration based on the results of the evaluation of current student engagement 
practices. The purpose of the recommendation will be to facilitate opportunities for 
faculty to assist students in discovering ways to become more involved in the classroom 
and engage assignments with vigor. 

Research Questions 

To conduct the student-engagement evaluation, five research questions were 
established. The purpose of the questions was to provide direction for the study to 
evaluate the student engagement process at the college: 

1. What current engagement activities are available for criminal justice students at 
the college? The rationale for this question was to identify existing practices and to 
establish a foundation for the identification of a model of student engagement activities. 

2. What models are identified in the literature of expected outcomes for best 
practices of successful student engagement processes and programs? The rationale for 
this question was to determine potential engagement practices to determine appropriate 
practices to be included in the recommendation to the criminal justice dean and the 
executive assistant to president/chief academic officer. 

3. What are the perceptions of students and instructors regarding student 
engagement at the college? The rationale for this question was to obtain feedback from 
students and faculty regarding their opinions of student-engagement practices currently 
used by the division. 

4. What valid components are necessary to formulate an evaluation tool that will 
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assess the worth of the engagement practices currently offered to criminal justice 
students? The rationale for this question was to construct a valid evaluation tool. 

5. What would be included in a valid recommendation to revise, implement, and 
evaluate an improved academic engagement process? The rationale for this question was 
to provide a viable recommendation regarding student engagement to the executive 
assistant to the president/chief academic officer and the criminal justice dean. 

Study Outcomes 

Five outcomes were associated with this applied dissertation study: 

1. Determine what engagement activities are currently being used in the criminal 
justice department. 

2. Identify the presence of any gaps between what the department is currently 
using for engagement activities, and what additional activities are available. 

3. Develop a valid evaluation tool. 

4. Provide a recommendation of engagement activities for future consideration. 

5. Submit the final report to the executive assistant to the president/chief 
academic officer and the dean of criminal justice. 

Dejin ition of T erms 

For the purpose of this study, the following terms necessitate additional 
explanation and are identified and defined. 

Advising hours. As part of their workload, instructors are assigned scheduled 
advising hours outside of instructional time. During this advising time, an instructor is 
expected to be available to students for academic and career advising. This time 
facilitates engagement opportunities between faculty advisors and students. 

Anclragogy. Andragogy is an instructional theory that focuses on the needs of 
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adult learners. Mezirow’s (1981) definition was, “Andragogy is an organized and 
sustained effort to assist adults to learn in a way that enhances their capacity to function 
as self directed learners” (p. 21). 

Cocurricular activities. Cocurricular activities were previously known as 
extracurricular activities. Such activities are created for students by educational 
institutions to promote a variety of activities encouraging student interaction in 
intellectual and social environments. These activities are normally linked to course 
curriculum. 

Cohort. This term refers to a group of students who form an affiliation while 
perusing an educational goal. This group enrolls in a common series of courses to form a 
community of learners. 

Commuter college. A commuter college is a college where the students do not 
reside on campus. The lack of residential facilities requires students to commute in order 
to attend classes at the college. 

Displaced student. A displaced student is one who enrolls or returns to college 
following a crisis event in his or her personal life. The event could include loss of a job, 
divorce, or geographic relocation. 

Engagement. This term refers to student involvement in the learning process. 
Engagement includes preparation for class, out-of-class discussions, and any other social 
or academic practice or activity contributing to the learning of academic material. 

Greater college community. A college represents a small component of a 
community. The greater community embraces elements of a community external to the 
college but that influence the college through financial and political means. 

Learning community. Gabelnick, MacGregor, Mathews, and Smith (1990) 
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characterized the creation of a learning community as “purposefully restructuring 
curriculum to link together courses or course work so that students find greater coherence 
in what they are learning as well as increased intellectual interaction with faculty” (p. 5). 
Learning communities may also be established through service-learning projects. 

Linked courses. Linked courses involve the pairing of two academic courses that 
complement one another. The courses may be of different disciplines but are thematically 
linked (Upcraft, Gardner, & Barefoot, 2005). 

Nontraditional student. A nontraditional student begins or returns to college later 
in life. The nontraditional student may be a lifelong learner, a displaced student, or any 
individual who is not a recent high school graduate. 

Office hours. As part of their workload, instructors are assigned scheduled office 
hours outside of instructional time. Such hours represent time available for students to 
schedule an appointment with the instructor to discuss general issues associated with their 
coursework. 

Practice. A practice is an activity customarily performed in a systematic manner 
and designed to lead to proficiency. 

Process. A process is a continuous operation or treatment established through a 
usual and routine set of practices and is also a group of related individual practices. 

Primary instructor. This term refers to a member of the criminal justice faculty. A 
primary instructor may be employed on a full-time basis or may be an adjunct faculty 
member so long as he or she is identified as being responsible for instruction as well as 
course curriculum development. 

Sendee learning. Service learning is an instructional strategy linking community 
service with classroom instruction. Goals of this pedagogy are to develop critical thinking 
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skills and a sense of social responsibility (Hubbert, 2002). 

Stakeholders. Stakeholders include a broad spectrum of individuals in the public. 
The Joint Committee on Standards for Educational Evaluation Programs (1994) defined 
stakeholders as “individuals or groups who may be affected by program evaluation” 

(p. 209). 

Traditional students. Recent high school graduates between 18 and 22 years of 
age who are enrolled full-time in a college are considered traditional students. Such 
students are enrolled without having entered the full-time workforce prior to enrollment. 

Urban commuter college. An urban commuter college is a postsecondary 
institution located in an urban setting. The majority of students reside off campus in the 
community, minimizing the necessity for the college to maintain residential facilities for 
students. 

Utility. The concept of utility focuses on the usefulness of a tangible or intangible 
item. The Joint Committee on Standards for Educational Evaluation Programs (1994) 
defined utility as “the extent to which an evaluation produces and disseminates reports 
that inform relevant audiences and have beneficial impact on their work” (p. 210). 




19 



Chapter 2: Review of the Literature 

Introduction 

This chapter focuses on literature related to the academic engagement of college 
students. The intent of this chapter was to discuss literature related to student engagement 
and practices that enhance students’ motivation to be more engaged with academic course 
work. 

Student Engagement 

The topic of student engagement by a 2-year college is difficult to isolate from 
that of 4- year colleges. Schreiber and Shinn (2003) explained this reasoning in providing 
the rationale for conducting their study of epistemological beliefs at a community 
college. The researchers noted that 44% of students who seek a degree attend community 
colleges, but only 5% of the research focuses on that group. 

The concept of engagement relates to students being actively involved with their 
studies both in and out of the classroom. Two national studies examined the concept of 
student engagement. The first is the National Survey of Student Engagement (NSSE). 

The survey is administered to first- semester and graduating students at 4-year colleges. 
The CCSSE is an equivalent survey administered to students attending 2-year colleges. 
Both surveys provided insight into the student engagement concept. 

In November 2004, the Community College Leadership Program released the 
results of the CCSSE. The report, titled Engagement by Design: 2004 Findings , critically 
reviewed the performance of 2-year colleges (as cited in Evelyn, 2004; McClenney, 

2004; Pekow, 2004). The report cited findings by the National Center for Education 
Statistics that a significant percentage of community college students failed to achieve 
their academic goals from 1995 through 2001. The number of students who reported 
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failure to achieve academic goals during the period of the study was appreciable. The 
report noted that only 25% of the students reported achieving their goal of transferring to 
a 4- year institution. The report also stated that, whereas “53% of the students cited 
transferring to a four-year college as a primary goal, an additional 21% name it as a 
secondary goal” (p. 3). Perhaps more important, statistics from the American Council on 
Education (2001) indicated that only one quarter of the students enrolled in public 2-year 
institutions actually graduated or received the desired certificate. The report is replete 
with examples of how the cultivation of student engagement would improve the quality 
of the 2- year students’ academic experience. 

Although the CCSSE report illustrates the necessity for colleges to improve 
student engagement, the question remains, how is engagement defined? Dictionaries 
contain many varied definitions for the concept of engagement. These definitions include 
military encounters, agreement to meet at a predesignated place, and premarital 
commitments. The inclusion of the term engagement in Merriam Webster’s Collegiate 
Dictionary in 1977 indicates that student engagement is not a new phenomenon, existing 
for over 30 years. Although the term engagement was defined in 1977, the origin of the 
term dates back to the 16th century. Analogous terms such as involve and connect date 
from the 14th and 15th centuries respectively. All three terms use or allude to the words 
interlock, come together, pledge, and participate within their definitions. The concept of 
engagement is centuries old. 

Mosenthal (1999) quoted Dewey, noting that the current concept of engagement— 
or as Dewey referenced it, reflective inquiry— originated as early as the 1920s. 
Engagement has also been identified by the term collaborative learning (Bruffee, 1999). 
The concept of engagement was an issue for academic scholars even before the NSSE 
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ignited scholarly discussion and scholarly research on the topic. 

The educational definition of engagement employs the term in a manner that 
relates commitment to the learning process. Various dictionary meanings help define the 
broad scope of the term. All of the definitions embrace commitment as part of the core 
learning process. The term commitment is a promise to participate in some undertaking 
(Brown, 1993) or a pledge to be involved in an activity. The term can also be related to 
pledging or assuming an obligation. Astin (1985) defined student engagement as student 
involvement, referring to “the quantity and quality of the physical and psychological 
energy the student invests in the college experience” (p. 156). Astin (1996) also discussed 
elements associated with student engagement. One of Astin’s goals was to define the 
concept of student engagement. In proffering his thesis, Astin linked the terms student 
involvement and engagement and essentially used them interchangeably. 

Scholarly writers reviewed the concept along with the dictionary definitions 
(Fredricks, Blumenfeld, & Paris, 2004) and provided a refined focus of the concept of 
engagement and student learning. Young (2003) wrote that, whereas engagement varies 
with course majors, engagement is essentially “how involved students are in the material 
they study and in the learning process itself’ (p. A37). This definition reveals that 
engagement is not the exclusive responsibility of the learner. The learning materials 
presented, along with the instructor’s class presentation, play a role in enabling the 
instructor to be a de facto participant in the student engagement process. 

Skinner, Wellborn, and Connell (1990) defined the term engagement as 
“children’s initiation of action, effort, and persistence on schoolwork, as well as their 
ambient emotional states during learning activities” (p. 24). Although this article focused 
on primary and secondary students, a correlation to technical and first-year college 
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students was found. Kuh (2003), in discussing the results of the NSSE, defined student 
engagement as “the time and energy students devote to educationally sound activities 
inside and outside the classroom and the policies and practices that institutions use to 
entice students to take part in activities” (p. 25). Kuh’s definition showed that 
engagement is not exclusively a student responsibility. Institutions, not just faculty, 
possess the ability, as well as the responsibility, to affect student-engagement levels 
during policy formulation. 

Fredricks et al. (2004) cited the work of Newmann, Wehlage, and Lamborn and 
the work of Wehlage, Rutter, Smith, Lesko, and Fernandez in discussing the 
psychological component related to student engagement. Both groups of authors wrote 
that a psychological investment in efforts directed toward the mastery of knowledge was 
necessary. Fredricks et al. (2004) considered Newmann’s and Wehlage’s psychological 
component in their examination of student engagement. Fredricks et al. noted that 
engagement was comprised of three learning elements: affective, cognitive, and 
behavioral. Other writers focused on two of these elements: behavior (being involved) 
and cognitive (being mentally invested in learning). No author has effectively integrated 
all three elements. 

Skinner et al. (1990) and Fredricks et al. (2004) maintained that the concept of 
engagement is primarily behavioral, but cognitive and affective components are 
identifiable. The concept of engagement having qualitative, cognitive, and affective 
components and quantitative, behavioral, and affective components link the writings of 
Astin (1996), Fredricks et al. (2005), Kuh (2003), Skinner et al. (1990), and Young 
(2003). The term possesses a meaning relating to behavior, emotion, and cognition. 

For the purpose of this dissertation, this researcher defines student engagement as 




23 



the amount of time a student devotes toward acquiring mastery of a skill or knowledge. 
The efforts dedicated towards learning occur any time within any academic, cocurricular, 
or social environment. 

Student Engagement Factors 

Several factors affect student engagement: student background, faculty 
involvement, institutional support of learners, and academic challenge. Student 
background acknowledges three influences affecting student engagement: familial and 
generational, lack of residential facilities, and student socioeconomic conditions (Pike & 
Kuh, 2005). Students who are the first in their families to attend a postsecondary 
institution face issues that may affect their level of engagement as well as academic 
success. Pike and Kuh studied student engagement of first- and second-generation college 
students and reported, “Unfortunately, a disproportionately low number of first- 
generation students succeed in college. ... A 15% gap exists between the 3-year 
persistence rates of first- and second- generation students (73% and 88% respectively)” 

(p. 276). Similarly, McClenney (2004) observed that many high school students were 
simply ill prepared for the rigors of college-level study. The observations by Pike and 
Kuh and McClenney suggest that many high school students lack the academic 
engagement skills for preparation for college-level intellectual study. 

Further complicating the student retention and engagement issues is the fact that 
many students attending a 2-year college are the first in their families to attempt a college 
degree. Pike and Kuh (2005) suggested that students who are the first in a family to 
attend college might require assistance in becoming engaged in their studies. The lack of 
experience in this area translates into the family’s inability to empathetically support the 
student, requiring additional assistance from faculty. First-generation students, lacking 
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additional assistance, may struggle through the collegiate acculturation process. Pike and 
Kuh explained as follows: 

Helping those who are first in their families to go to college is challenging for 
many reasons, one of the more important of which is that many first-generation 
students do not engage in the wide range of academic and social activities that the 
research shows associated with success in college, (p. 292) 

The responsibility for fostering student involvement and engagement within the first term 

is transferred to members of the faculty. 

ASU (n.d.a) identified benefits for both students and faculty who engage in 

external class discussions. The benefits were recognized in the works of several 

researchers (Jaasma & Koper, 2001; Kuh & Hu, 2001; Plecha, 2002). The first benefit 

that students exhibited was an increased confidence in their scholastic abilities. 

Instructors possessed the opportunity through informal interaction to praise the 

demonstrated effort of a student. Second, improvement in academic performance was 

noted. Students who were struggling with a concept gained the opportunity to discuss the 

idea with the professor. The conversation could provide the instructor with an 

opportunity to communicate on a level the students would more readily understand. 

Third, students demonstrated greater effort in academic activities that they perceived as 

purposeful. Students could gain insight from the professor as to why an assignment or 

activity was considered important. Fourth, students who engaged in cocurricular 

discussions with faculty enjoyed increased persistence rates. Educators took the 

opportunity to reinforce the students’ ability to be successful in the course. Fifth, this 

group of students was more likely to desire and to pursue graduate-level studies. 

Instructors were able to grasp the opportunity to reinforce that students possess the 

intellect to successfully study at the graduate level. Sixth, the students integrated into the 
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college environment, both social and academic, better than students who did not 
communicate with faculty. Faculty possessed an opportunity to direct students toward 
activities the students might find rewarding, both college and community based. The 
researchers indicated that, in order to provide the most meaningful academic direction, 
the instructor-student relationship has to be more individualized than the classroom 
allows. 

ASU (n.d.b) listed a series of tips for students to help improve interaction between 
faculty and students. For a variety of reasons, instructors frequently intimidate students. 
The tips supplied provide suggestions to assist students in overcoming this sense of 
intimidation, allowing for enhanced student-faculty engagement and substantive 
interaction (see Appendix D). 

Faculty Involvement in Student Engagement 

Barr and Tagg (1995) observed that a new model was replacing the traditional 
instructional standard. In discussing the traditional standard, they noted, “Under it, 
colleges have created complex structures to provide for the activity of teaching conceived 
primarily as delivering 50-minute lectures— the mission of a college is to deliver 
instruction” (p. 13). College administrators recognized that their missions had shifted 
from delivering instruction to promoting learning. The promotion of learning is the 
learning model. Even though the subject college has attempted to encourage use of the 
learning model, use of the traditional instructional standard persists. 

Undertaking engagement in learning opportunities is no longer considered the 
exclusive responsibility of students. Several researchers focused on the role of the faculty 
in student learning engagement (Handelsman, Briggs, Sullivan, & Towler, 2005; Kuh, 
Laird, & Umbach, 2004; Reed, 1989; Umbach & Wawrzynski, 2005). Reed maintained, 
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“There are two elements essential to being a good teacher: enthusiasm for the subject and 
a genuine interest in the personal, as well as the professional, well being of the students” 
(p. 555). The two factors identified by Reed, when combined with well-crafted learning 
activities, produce the potential to create an enthusiasm within students that will inspire 
students to do their best in their pursuit of knowledge. 

Handelsman et al. (2005) conducted two independent studies regarding student 
engagement. Both studies reported findings consistent with other previously published 
research results. The Handelsman et al. research also produced the finding that faculty 
members play a significant role in developing student engagement in the classroom: 

This research supports a multidimensional construct of student engagement. 

Moreover, some of the dimensions are not necessarily obvious to the observers. 

Thus, instructors receive only part of the picture if they focus on the obvious signs 

of engagement, such as raising hands and asking questions, (p. 190) 

This statement supports the association of student engagement with the 
andragogical principle that learning occurs in any context and at any time. The statement 
also reinforces the observation that proactive faculty members search for signs that 
students are engaged in their studies and are learning. Handelsman et al. (2005) discussed 
the results of their research: “Participation/interaction was the factor that related to a 
variety of internal and external indexes. That finding is consistent with previous research 
. . . that highlights the importance of faculty interaction and learning” (p. 189). 

Turner and Patrick (2004) supported the theory that students require assistance in 
order to become engaged learner: “Work habits, like other behaviors, arise from the 
interaction of both characteristics of students and of the learning environment” (p. 1759). 
Given the real and potential impact of faculty members on the learning environment, their 
contributions to student engagement ought not to be overlooked or overstated. 
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Umbach and Wawrzynski (2005) examined the role of faculty as related to 
student engagement. Their research utilized data from two sources. The first was the 
2003 NSSE. The second was a survey of faculty members at institutions participating in 
the 2003 NSSE. These researchers concluded, “We found that faculty behaviors and 
attitudes profoundly affect students, which suggests that faculty members may play the 
single most important role in student learning” (p. 176). This work further demonstrates 
the profound impact by faculty on student learning. 

Astin (1996) identified three potent forms of student engagement. Two of the 
forms are clearly affected by instructors: student-academic involvement and 
student-faculty involvement. In each, the instructor is a key participant. In the first 
instance, the instructor designs and implements an academic program in a manner that 
encourages and assists a student to focus intently on learning the subject matter. In the 
second instance, the instructor meets individually with students. In a best-case scenario, 
the meeting results in building a rapport that inspires students to learn and encourages 
further future interaction. 

Astin (1996) identified a third form of student engagement. This form is not 
readily identified as was improving faculty and student interaction with peer groups. This 
third type of engagement is assisting students to identify cocurricular activities that relate 
to a student’s individualized interests. Although the third variety of student engagement 
appears unrelated to faculty influence, this is not necessarily an accurate observation. The 
influence of a respected faculty member’s support of specific cocurricular programs 
cannot be precisely measured. Through a student-centered teaching orientation, faculty 
members positively influence students in a direction that enhances their opportunities to 
collaboratively engage others in their quest for knowledge. 
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Astin (1985) noted, “Community colleges are places where the involvement of 
both faculty members and students appear to be minimal: All students are commuters and 
most are part-timers” (p. 146). This observation continues to be accurate more than 20 
years later. Faculty members are reluctant to change from traditional instructional 
activities to being active participants in facilitating student learning. Instructors who 
adopt andragogical teaching methods make that transition by becoming active 
participants in student engagement practices. Astin (1985) described an engaged faculty 
member: 

A highly involved faculty member would put an appreciable amount of time and 
energy into teaching activities, regularly seek out student advisees to monitor their 
progress, actively participate in departmental and institutional functions, and 
make a conscious effort to integrate research and teaching activities, (p. 134) 

McClenney (2004) suggested that a disconnect exists between the faculty’s 

perception of student interaction and reality: 

So when data suggest that faculty- student interaction is less extensive than faculty 
members believe it is— which often happens— they begin to realize that their 
impressions of the quality of education at their college are based on the best 
student experience rather than the typical student experience, (p. 5) 

Weimer (2002) made a similar observation regarding faculty and student perceptions 

noting, “Faculty consistently say students are more than students report they are” (p. 73). 

Astin (1985), McClenney, and Weimer all noted that a disconnect exists between faculty 

and students. This disconnect may be attributed to the inefficiencies associated with the 

evaluation of student engagement. 

Samuelowicz and Bain (2001) noted that a change in teaching orientation allows 
faculty to focus on improving the typical student experience. Various instructional 
orientations run from the simple transmissive course lecture to a method involving 
student inputs. The latter facilitates learning. The facilitated orientation is a practice that 
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most encourages student engagement. Samuelowicz and Bain stated, “Academics 
expressing such an orientation were in transition from a transmissive to a facilitative 
perspective, and that interaction between the student and teacher was the active agent in 
the change process” (p. 301). Once the orientation process has transitioned, increased 
student engagement may be expected to follow. 

Engagement has been tied to student success and retention. Rhodes and Nevill 
(2004) discussed retention and student success, blending the concepts of academic and 
social integration. “Academic integration includes academic performance, 
self-perceptions, academic progress and belief that lecturing staff are personally 
committed to teaching and supporting students. Social integration includes self-esteem 
and the quality of relationships established with teaching staff and peers” (p. 181). 

Rhodes and Nevill suggested that a linkage between student success and student retention 
exists and extends to student-faculty engagement. This study supports the concept that 
facilitated learning inspires students to engage their studies, resulting in a positive 
collegiate experience both academically and socially. 

College policies also affect student engagement. An administration that fails to 
consider the impact of college policies on education risks compromising the quality of 
student engagement and learning. Haworth and Conrad (1997) held, “Faculty and 
administrators [who] invest in teaching and learning activities— such as ‘real world’ 
lectures and hands on learning— . . . invite connections between theory and practice” (p. 
34). 

Colleges are obliged to promulgate policies in support of andragogical activities. 
Policymakers who fail to take into account student engagement activities during policy 
formulation also risk alienation of both students and faculty. Walters (2003) pointed out 




30 



that, with regard to student retention, instructional methods were as important as 
academic support systems in assisting students in meeting their goals. 

Student Engagement in a Criminal Justice Program 

A dearth of information exists specifically relating the concept of student 
engagement and adult learning to technical colleges or criminal justice programs. Three 
articles were located relating to andragogy and criminal justice instruction. Birzer (2004) 
discussed the application of Knowles’ seven principles of andragogical practice in 
collegiate criminal justice studies. Finckenauer (2005) discussed the quality aspect of 
criminal justice programs. Finckenauer expressed that, historically, law enforcement 
education lacked credibility. As law enforcement evolved into a profession, a 
proliferation of collegiate criminal justice programs was spawned. This sudden demand 
for faculty resulted in less-than-adequately prepared instructors. While these instructors 
were experienced practitioners, they lacked academic credentials. The result was 
marginal curriculum and poor assessment processes. This compromised the expected 
rigor of college-level study. 

Sims (2006) described the impact and application of active learning techniques in 
creating an active learning environment in collegiate criminal justice programs. Sims 
contended that active learning techniques were quite applicable to criminal justice 
studies. These techniques engage students, stimulating their critical thinking skills. Sims 
explained that there are four active learning techniques, and she compared the advantages 
and disadvantages of those techniques (see Appendix F). 

Birzer (2004) advocated applying Knowles’ seven andragogical practices to 
criminal justice education. Birzer listed the seven principles: (a) establish a physical and 
psychological climate conducive to learning, (b) involve learners in mutual planning of 
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methods and curricular corrections, (c) involve learners in diagnosing their learning 
needs, (d) encourage learners to formulate their own learning objectives, (e) encourage 
learners to identify resources and to devise strategies for using such resources to 
accomplish their objectives, (f) help learners to carry out their learning plans, and (g) 
involve learners in evaluating their learning. 

Birzer (2004) also identified four advantages of the application of the 
andragogical approach to criminal justice education: “(1) it draws on students’ past 
experiences, (2) it treats students as adults, (3) it adapts to the diverse needs and 
expectations of students, and (4) it develops critical thinking, judgment, and creativity in 
the learner” (p. 398). Each advantage contributes to the effectiveness of criminal justice 
learning. Birzer identified the fourth advantage as the most important. Within the context 
of vocational or community college instruction, the fourth advantage is critical. Many 
students graduate from 2-year institutions at the relatively young age of 19 to 22 years. 
Such students often lack the life experience that hones judgment, critical thinking, and 
problem-solving skills. For this reason, incorporating andragogical principles to attain the 
identified advantages in the classroom is important, if not vital. Table 2 illustrates the 
correlation of Knowles’ seven andragogical principles to the principle learning 
advantages as described by Birzer. 

Finckenauer (2005) identified several problems that were associated with criminal 
justice programs. Deficiencies exist in criminal justice education; the deficiencies cited 
included lack of course rigor, poor evaluation methods, and curriculum issues. These 
deficiencies Finckenauer noted are historical. Criminal justice as an academic discipline 
is relatively new. The call for a better educated law enforcement officer dates back to the 
1931 Report on Police commissioned by the Wickersham Commission on Law 
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Observance and Enforcement. The commission was charged with examining prohibition 
enforcement and the repeal of the 18th Amendment. One finding of the commission was 
related to police education and training. The commission noted the importance of police 
being educated. 

Table 2 

A Comparison of Andragogical Principles to Learning Advantages 



Andragogical principle 


Principle advantage 


Establish a physical and psychological 
climate conducive to learning. 


Facilitates student engagement. 
Identifies the diverse needs of learners. 


Involve learners in mutual planning of 
methods and curricular corrections. 


Facilitates student interaction. 
Treats students as adults. 

Begins the critical thinking process. 
Acknowledges student experiences. 


Involve learners in diagnosing their learning 
needs. 


Facilitates student engagement. 
Treats students as adults. 

Begins the critical thinking process. 
Acknowledges student experiences. 


Encourage learners to formulate their own 
learning objectives. 


Facilitates student engagement. 
Treats students as adults. 

Begins the critical thinking process. 
Acknowledges student experiences. 


Encourage learners to identify resources and 
to devise strategies for using such resources 
to accomplish their objectives. 


Facilitates student engagement. 
Treats students as adults. 

Begins the critical thinking process. 
Acknowledges student experiences. 


Help learners to carry out their learning 
plans. 


Facilitates student engagement. 
Begins the critical thinking process. 


Involve learners in evaluating their learning. 


Facilitates student engagement. 
Treats students as adults. 

Begins the critical thinking process. 
Acknowledges student experiences. 
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In 1967, the topic of police education came to the attention of policymakers. The 
President’s Commission on Law Enforcement and Administration of Justice (1967) 
called for police officers to earn a baccalaureate degree. At the time of the report, the 
average educational level of a law enforcement officer was 12.4 years (Schmalleger, 
2005). 

Little attention was directed toward improving law enforcement education until 
the 1960s. Totzke (1999) acknowledged the educational efforts of Berkeley Police Chief 
A. Vollmer and Federal Bureau of Investigation Director J. Edgar Hoover during the 
1930s. Although such efforts were notable, they were nascent. Vollmer was among the 
first police chiefs to acknowledge the importance of a well-educated police force. Deakin 
(1988) explained that Vollmer recruited entry-level officers from the University of 
California at Berkeley. Deakin quoted Vollmer, who identified the importance of proper 
personnel selection quoting the Maxims of Confucius: “The successful administration of 
any government depends entirely upon the selection of proper men” (p. 95). Vollmer was 
also recognized as initiating one of the first police academy training programs in 1908 
(Deakin; Totzke). 

De a kin (1988) reported that Vollmer was credited with being instrumental in the 
development of the first law enforcement college curriculum. Criminal justice as a minor 
course of study was first offered at the University of California at Berkeley in 1923. The 
first associate degree program was offered at San Jose College in 1930 and identified the 
importance of the application of various sciences to criminal investigations. 

Similarly, the Federal Bureau of Investigation under Hoover identified a need for 
improved law enforcement training. The National Academy was developed in 1935 to fill 
this need (Totzke, 1999). The mission of the National Academy was to train command 
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officers on the latest law enforcement administration and investigation techniques. 
However, little consideration was given to the need for improved entry-level training for 
law enforcement officers. 

The Fifth Circuit Court of Appeals endorsed the concept of minimum police 

education. The court upheld a Dallas, Texas, requirement that an entry-level police 

applicant must possess 45 college semester credits with a C average. In the matter of 

Davis v. City of Dallas (1985), the court held as follows: 

.The position of officer on the Dallas police force combines aspects of both 
professionalism and significant public risk and responsibility. We regard this 
distinction as crucial, and affirm the District Court’s finding that the appellees’ 
education requirement bears a manifest relationship to the position of police 
officer, (p. 226) 

A proliferation of criminal justice programs occurred between 1967 and 1977. 

The increase was attributable to three government actions. The first was the 1965 Law 
Enforcement Act. The second was a report, The Challenge of Crime in a Free Society 
(President’s Commission on Law Enforcement and Administration of Justice, 1967), that 
contributed to the 1968 Omnibus Crime Control and Safe Streets Act. This act enabled 
the commission of the Law Enforcement Assistance Act and the Law Enforcement 
Education Program. The establishment of the Law Enforcement Education Program was 
the third action that triggered the expansion of criminal justice programs. This program 
funneled funding to colleges, police agencies, and individual officers, enabling officers to 
return to college. Officers returning as students resulted in an increase of criminal justice 
programs, exploding from 209 in 1967 to 1,245 in 1977, an increase of 600% (Deakin, 
1988; Totzke, 1999). 

The use of andragogical principles minimizes, if not neutralizes, the lack of 
course rigor, poor evaluation methods, and curriculum issues that Finckenauer (2005) 
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identified. Student participation, where feasible, will assist instructors in developing a 
challenging curriculum with valid evaluation activities. The impacts of program growth 
issues, present since September 11, 2001, are minimized by controlling the learning 
environment. 

The inclusion of andragogical practices also neutralizes another issue Finckenauer 
(2005) identified. This issue was identified as faculty qualifications. Faculty and 
instructors who possess appropriate vocational experience and academic credentials and 
employ andragogical principles will perform at an above-average level. 

Sims (2006) wrote that a learning environment is enhanced by using student- 
engagement activities. Sims expressed that “a student-centered learning environment 
would fit in criminal justice classrooms given the ‘hot topic’ issues that provide fodder 
for robust learning activities” (p. 336). The article identified four active learning 
andragogies that are useable in the classroom” collaborative learning, problem-based 
learning, service learning, and the involvement of students in research. Sims 
acknowledged that each andragogy possessed advantages and disadvantages (see 
Appendix F). 

Andragogy and Student Engagement 

Two definitions of andragogy are proffered for this study. The first definition is 
provided by Knowles (1970), who defined andragogy as the “art and science of helping 
adults learn” (p. 38). The Knowles definition recognizes andragogy as a legitimate 
approach to educating adult learners. The definition also suggests that instructors 
personalize their instructional presentation for each group of students. 

The second definition of andragogy is provided by Mezirow (1981), who stated, 
“Andragogy is an organized and sustained effort to assist adults to learn in a way that 
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enhances their capacity to function as self directed learners” (p. 21). Mezirow supports 
Knowles’ concept of andragogy as a science. This defines, or redefines, the role of the 
instructor in terms of a facilitator for learning rather than an omniscient person holding 
the keys that permit access to information and learning. 

Learning Communities in Student Engagement 

Learning communities play a role in fostering student engagement in their 
academic and social lives. The concept of learning communities found basis in the work 
of Meiklejohn and Dewey from the 1920s and 1930s (Kellogg, 1999; Minkler, 2000; 
Price, 2005; Smith, 2003; Webber, 2000). Dewey focused on experiential learning. 
Minkler noted that Dewey agreed with a philosophy of Thomas Jefferson that one goal of 
education was to promote a sense of democracy in students. Minkler stated that Dewey 
believed the best way to accomplish this goal was through a “community of learners 
working collectively to solve problems” (p. 2). Minkler also noted that Dewey agreed 
with Aristotle that students develop an “important relationship between the use of the 
learner’s reason with that of the learner’s actual experience” (p. 15). Minkler pointed out 
that Aristotle in Nicomachean Ethics discussed how “knowledge could neither be said to 
be all logos (reason) nor could it be said to be all praxis (experience)” (p. 12). 

Kellogg (1999) related that Meiklejohn developed the Experimental College at the 
University of Wisconsin in 1927, and although the college existed for only 6 years, 
researchers acknowledge that this was the initial establishment of learning communities 
as we know them today. The concept of the Experimental College was premised upon the 
integration of a learning theme across all courses instructed in a curriculum. Price (2005) 
noted that “Meiklejohn focused the college on the principles of integrated learning with a 
common reference that dissected all aspects of a diverse curriculum taught by many 
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faculty members” (p. 5). Smith (2003) affirmed that Meiklejohn articulated that teaching 
students how to think was more important than teaching students what to think. Smith 
further noted that the educational system Meiklejohn sought to develop fostered civic 
engagement both in the classroom as well as outside the classroom. 

Webber (2000) discussed research conducted in 1969 by Tussman. Tussman, a 
student of Meiklejohn, was dissatisfied with liberal arts education, believing such an 
education had an adverse effect on students’ ability to understand human nature. Webber 
quoted Tussman, who advocated learning communities consistent with the Experimental 
College concept: 

The course forces teaching into small, relatively self-contained units. 

Horizontally, courses are generally unrelated and competitive. . . . No teacher is in 
a position to be responsible for . . . the student’s total educational situation. The 
student presents himself to the teacher in fragments, and not even the advising 
system can put him together again. . . . Horizontal competitiveness and 
fragmentation of student attention are limiting conditions of which every sensitive 
teacher is bitterly aware. But there is nothing he can do about it. He can develop a 
coherent course, but a collection of coherent courses may be simply an incoherent 
collection. For the student, to pursue one thread is to drop another. He seldom 
experiences the delight of sustained conversations. He lives the life of a distracted 
intellectual juggler, (p. 21) 

Opportunities for student engagement increase when courses are integrated with 
other diverse course offerings. Learning communities encourage the integration of 
thematically linked courses that allow students to observe how courses interact with each 
other and potentially contribute to deeper understanding and real-world application of the 
material (Gabelnick, MacGregor, Mathews, & Smith, 1990). 

Upcraft et al. (2005) identified that learning communities continue to be 
represented in the collegiate setting. The use of learning communities is applicable in any 
collegiate learning environment. Associate degree programs use learning communities 
approximately 20% of the time, whereas the percentage of use escalates to 75% by 
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research-intensive institutions. 

Several researchers examined the importance of learning communities. Cross 
(1998) discussed learning communities and defined the term as “groups of people 
engaged in intellectual interaction for the purpose of learning” (p. 4). Smith and Hunter 
(1988) defined a learning community as curriculum structured in a manner in which 
students are actively engaged in a sustained academic relationship with peers and faculty 
for a longer duration than a traditional course of study would allow. A common definition 
cited in the literature is by Gabelnick et al. (1990), who defined a learning community as 
“purposefully restructuring curriculum to link together courses or course work so that 
students find greater coherence in what they are learning as well as increased intellectual 
interaction with faculty” (p. 5). 

Upcraft et al. (2005) used the 2001 MacGregor, Smith, Matthews, and Gabelnick 
definition of a learning community: 

A variety of approaches that link or cluster classes during a given term, often 
around an interdisciplinary theme, that enroll a common cohort of students. This 
represents an intentional restructuring of students’ time, credit, and learning 
experiences to build community among students, among students and their 
teachers, and among disciplines, (p. 372) 

The thread of commonality running through these notable definitions is that each 
definition groups students together in a structured course curriculum. The structured 
curriculum facilitates learning with increased interaction between students, their peers, 
and faculty members. 

The objective of learning communities is to mesh students into communities 
where their academic and social lives will facilitate learning. Zhao and Kuh (2004) 
reported the following: 

Findings from this study tend to corroborate previous research on the value added 




39 



effects of participating in learning communities. Learning communities are 
associated with enhanced academic performance, integration of academic and 
social experiences, gains in multiple areas of skill, competence and knowledge, 
and overall satisfaction with the college experience, (p. 131) 

The positive effects of learning communities reported by Zhao and Kuh (2004) 

are reinforced in the literature. Bowden, Hart, King, Trigwell, and Swamm (2003) made 

observations similar to those of Zhao and Kuh, as did Smith and Bath (2006). Bowden et 

al. commented that students were better prepared when presented with opportunities to 

leam in a situational environment. Upcraft et al. (2005) wrote that students learn “most 

effectively in the context of a compelling problem” (p. 374). Upcraft et al. reinforced 

other authors’ observations (Bowden et al.; Smith & Bath; Zhao & Kuh; Zhu & Baylen, 

2005). Students learning in a situational environment are afforded the ability to develop 

sensitivity to a myriad of situations. Smith and Bath (2006) cited Bowen, observing that 

college education “increases the capacity of people to tolerate, to understand, and to 

communicate with others. Indeed education is the most significant predictor of tolerance. 

. . . College reduces authoritarianism, dogmatism, ethnocentrism, and prejudice in the 

intellectual sphere” (p. 265). The latter traits are hardly conducive to a civic-minded 

citizen, let alone a successful law enforcement officer. 

Learning occurs neither through osmosis nor in a vacuum, and learning 

communities assist students in navigating complex issues. Learning communities 

empower students, allowing them to discover new ways to approach difficult and 

complex situations (Smith, 2003). Zhao and Kuh (2004) identified that learning 

communities also assist students in making connections: 

Learning communities are intentionally structured to help students make two 
types of connections consistent with this theoretical orientation. The first is 
encouraging students to connect ideas from different disciplines, which is aided 
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by being co-enrolled in two or more courses. . . . The second connection is the 
linking of students through ongoing social interaction afforded by being with the 
same students for an extended period of time. (p. 117) 

The connections noted by Zhao and Kuh are important contributions toward the 

development of collaboration between experiential and theoretical learning. 

Observations by Bowden et al. (2003) reinforce the importance of situational 

learning for law enforcement students. Traits that are increased are essential for any law 

enforcement officer to be successful. Those traits identified as being reduced are 

corrosive to a law enforcement career and interpersonal qualities society demands a 

professional law enforcement officer not demonstrate. The discussed traits are difficult to 

impact in a traditional transmissive learning environment. A transformed 

student-centered learning, community-oriented method would influence each behavior 

(Birzer, 2004; Bowden et al.; Finckenauer, 2005; Smith & Bath, 2006). 

Various implementation models describe or characterize learning communities. 

The literature identified four prominent models (Chaves, 2003; Kellogg, 1999; Price, 

2005; Upcraft et al. 2005). The four models are (a) linked or clustered courses, (b) 

cohorts in large classes, (c) team-taught programs, and (d) residence-based programs. 

Each model exhibits a specific orientation; however, they share the common goal of 

improved student learning. 

The first model is linking or clustering courses. Linked courses are simply the 
pairing of two courses that share common components and mutually reinforce one 
another. Clustering involves three or more classes centered on a common theme (Upcraft 
et al., 2005). Linked or clustered course model students are automatically enrolled in all 
course sessions, thereby building a cohort of students that becomes a learning 



community. 
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The second model is a cohort in a large class. Both Upcraft et al. (2005) and 
Kellogg (1999) referenced this model as freshman interest groups. This style of learning 
community features a “master learner” who serves as a course guide. Upcraft et al. 
identified a master learner as an instructor who takes the course and is responsible for 
assisting “. . . students [to] synthesize what they are learning” through a seminar session 
(p. 379). 

Dabney, Green, and Topalli (2006), in discussing freshman interest groups, wrote 
that students were more involved in student activities when the activities were associated 
with a learning community. Correspondingly, Hu and Kuh (2003) reported that students 
who are more engaged “in educationally purposeful activities” perform at a higher 
academic level than students who are uninvolved (p. 197). 

The third model is team teaching. Upcraft et al. (2005) noted that learning 
communities that are team-taught “are the most extensive in terms of curricular 
integration and faculty role” (p. 379). The comprehensiveness of this method is 
attributable to faculty collaboration in designing the curricula (Price, 2005). This learning 
community model requires that instructors focus on a cluster of courses with an 
interdisciplinary theme rather than a typical unilateral course focus. 

The fourth model is residence based. The rationale is for students to live in 
residential facilities in order to integrate experience with their academic studies. 
Schroeder (1994) wrote, “Learning communities are fostered by a commonality and 
consistency of purpose, shared values, and transcendent themes” (p. 380). The residence 
model reinforces values as well as the academic theme due to informal student 
interactions outside of formal academic meetings. 

Smith and Bath (2006) provided the following list of characteristics of a learning 
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community: 

A review of the literature shows that a learning community can be broadly 
characterized as a collective of learners in a learning context that emphasizes 
social interaction and identity over individual action (Wenger 1989) collaboration 
among students, active engagement in problem solving, the generation of 
solutions and research questions related to solutions related to the material being 
learned, . . . interactions between faculty and students as well as among students, 

. . . peer teaching and peer assessment, . . . and continuity of the learning cohort 
over time. (p. 267) 

Two other author dyads (Lenning & Ebbers, 1999; Oates & Leavitt, 2003) 
identified characteristics of learning communities. The authors respectively identified 
four and eight characteristics of learning community activities. The four activities 
identified by Oates and Leavitt were interdisciplinary, collaborative, thematic based, and 
team-taught activities. These activities intertwined with the activities identified by 
Lenning and Ebbers on multiple levels. The first activity noted by Lenning and Ebbers 
was organizing students and faculty into smaller groups. This activity corresponds to 
three of the four activities identified by Oates and Leavitt: interdisciplinary, 
collaborative, and thematic -based activities. Encouraging integration of the curriculum 
corresponds to collaborative, interdisciplinary, and team-taught activities. Helping 
students establish academic and social support networks is collaborative in nature. The 
provision of a setting for students to be socialized to the expectations of the college is 
both thematic based and collaborative. 

Linding more meaningful ways to bring faculty together correlates with 
team-taught activities. Locusing faculty and students on learning outcomes incorporates 
both collaborative and thematic-based learning activities. The provision of a setting that 
permits community-based delivery of academic support programs is an interdisciplinary 
approach to learning. Examining the first-year learning experience through a critical lens 
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is both collaborative and thematic based. 

Minkler (2000) observed that Dewey, Meiklejohn, and Aristotle all embraced the 
concept of learning as an experiential process. Learning communities are but one method 
to experience and address complex issues in an academic, or learning, environment. 
Evaluation of complex issues frequently requires a multidisciplinary approach by a group 
and is best embraced by a group of learners working within a structured and integrated 
curriculum. 

Engagement Practices 

Researchers who examined engagement discussed the concept of student learning 
and effective instructional practices being linked together (Bain, 2004; Kuh et al., 2004; 
Handelsman et al., 2005). Bain researched the practices of the best college instructors. 
Bain maintained that the instructors were focused on the art and science of teaching. One 
of the conclusions of Bain’s work was that “scholarship centered around four 
fundamental inquiries” (p. 49). Student engagement is encouraged and instruction 
improved by examining the manner in which course presentations are prepared. Bain 
identified four questions that scholars used to guide lesson development that focused on 
student-centered delivery: 

(1) What should my students be able to do intellectually, physically, or 
emotionally as a result of their learning? (2) How can I best help and encourage 
them to develop those abilities and the habits of the heart and mind to use them? 
(3) How can my students and I best understand the nature, quality and progress of 
their learning? and (4) How can I value my efforts to foster that learning? (p. 49) 

Each question focused on how the material or instructional approach would best aid the 

student in comprehending the course material. The developmental practices position 

student learners ahead of the professorial ego, further encouraging student engagement. 

One form of experiential learning for criminal justice students is the mock trial. 
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The mock trial requires students to be involved with the exercise and is a simulation of a 
real-world process (Kravetz, 2001). In simulating the trial, students are able to internalize 
learning, as they are directly involved in the process. Kravetz advocated the mock trial as 
an effective method of engaging students in their learning. Keeping the trial consistent 
with the academic abilities of the students is important. An uncomplicated and 
straightforward trial provides the most benefit to the students. 

Service learning is another student-engagement activity. The concept dates back 
to 1862. As one component of the Land Grant Act that established universities 
nationwide, service to the community was included as a primary collegiate mission 
(Penn, 2003). Boyer (1994) acknowledged that, in 1896, future President Woodrow 
Wilson stated, “It is not learning, but the spirit of service that will give a college a place 
in the annals of the nation” (p. 8). The statement is prophetic in that the concept of 
service learning matured over the next century into a respected pedagogical model. 
Wilson would, in 1902, become president of Princeton University before being 
inaugurated as the 28th President of the United States in 1913. 

The use of service learning increased during the 1980s. The increase was 
attributable to reports released by the Carnegie Foundation. The foundation reported that 
students were receiving inadequate citizenship training. Responsibility for the failure fell 
upon the educational system (Hirschinger-Bank & Markowitz, 2006). 

Service learning is the integration of out-of-class activities designed to 
complement course outcome achievement (Oates & Leavitt, 2003). Service learning is an 
extension of or is complimentary to learning communities. Hubbert (2002) endorsed 
service learning as an effective method to connect students with a prospective faculty 
member. This linkage helps to integrate a student’s “interpersonal development with their 
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academic and cognitive development” (p. 5). Another benefit of service learning is that 
students may critically review their role in the project assignment. Through this critical 
review, students reflect upon how the project activities influenced their interpersonal 
values (Hirschinger-Bank & Markowitz, 2006). 

Service-learning courses assist colleges in identifying community and vocational 
needs. This identification is often accomplished vicariously through the activities of 
service learning students. The identification allows real community and vocational needs 
to be met rather than having academia presume that students know those needs (Penn, 
2003). 

Service learning provides a beneficial component to a criminal justice curriculum. 
This pedagogical model provides students with opportunities to observe the criminal 
justice system in operation. With this insight, an opportunity exists for students to 
observe how various policy makers react to the issues and the impact of policy decisions 
on institutions and the public (Hartmus, Cauthen, & Levine, 2006). 

Penn (2003) discussed various benefits that are associated with service learning. 
First is the ability of students to “apply abstract principles to ‘real-world’ situations” (p. 
376). Bringle and Hatcher (1996) wrote that service learning infused a new vitality into 
the classroom. Students developed interest in the course, and their performance, as well 
as their ability to solve problems, improved. Service learning was also found to 
contribute to higher midterm and final exam grades (Markus, Howard, & King, 1993). 
Frederickson (2000) described how students self-reported improved understanding of 
subject matter from attending courses taught under the model of service learning. 

Criminal justice curricula traditionally utilized internship programs to provide 
students with a dose of real-world experience. Penn (2003) questioned the value of 
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internship programs and noted that students traditionally take the internship course during 
the final semester of classes. At that point in their education, students are focusing on life 
after college, which effectively overshadows the potential benefits of the course. Instead, 
students focus on the task of finding employment in a full-time position. 

Service learning is a method that adds an experiential component to students’ 
academic experience. This experiential component allows students to apply the cognitive 
component of their studies to real-world experience. Hubbert (2002) observed that this 
experiential component embraces the concept of developmental theorist Piaget, whom 
Hubbert said “argued that intelligence is shaped by experience” (p. 6). 

The term service learning has a plethora of definitions. As Hubbert (2002) 
remarked, common to all the definitions are elements of “learning and reflection that 
distinguish service learning from volunteerism or charity work” (p. 5). Hirschinger-Bank 
and Markowitz (2006) defined service learning within a criminal justice context. The 
definition was that service learning is an instructional model integrating scholastic 
learning with learning through community service. 

Several examples of effective learning programs initiated within criminal justice 
programs. Hirschinger-Bank and Markowitz (2006) evaluated one such program 
associated with juvenile justice. The program identified five outcomes: (a) achievement 
of diversity, (b) interpersonal development, (c) intellectual development, (d) personal 
development, and (e) citizenship behavior. The program was successful; students were 
able to grasp larger issues considered systemic to the juvenile justice system. 

A program at John Jay College is another example showing that service learning 
is beneficial to criminal justice students. The college offers seminars on judicial process 
courses in two different disciplines, the department of public management and the 
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government department (Hartmus et al., 2006). The component of this class that involves 
service learning involves students serving as court monitors. The arrangement between 
the college and the Fund for Modern Courts included a requirement that the monitors 
must be enrolled in an arduous academic course. This structure allowed students to 
discuss their experiences. The discussion component provided the opportunity for 
reflection, which is a characteristic of a course developed under the model of service 
learning (Penn, 2003). The intent of the course was accomplishment of two goals. The 
first goal was to orient students to the most complicated court system in the United 
States. The second goal was to facilitate the success of the Modern Courts monitoring 
program goals of educating the public about the court system and providing constructive 
feedback focused on making the courts more efficient (Hartmus et al.). 

Service learning is related to learning communities and is another practice that 
may increase student engagement. Service learning is also amenable to being linked with 
other engagement activities, such as journal writing. Journal writing would be beneficial 
when attached to service learning as the two practices are thematically linked. The learner 
is allowed to document experiences and reflect on the value of such experiences 
(de Acosta, 1995). 

De Acosta (1995) studied the impact of journal writing as a method of engaging 
students in experiential learning activities and reported, “Journals provide students with a 
space for active, regular, systematic thinking about what they are learning. In service 
learning courses, where keeping journals is a fairly common practice, journal writing 
contributes to integrating experiential learning with classroom learning” (p. 141). In 
addition to engaging in the experience of service learning, the journal exercises also 
allow the students “to step back from the learning experience to record it, speculate about 
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it, reflect about it, connect it to theories and concepts learned in the course, and make 
sense of it in preparation for speaking to others in class and writing papers” (p. 142). 
Standards of Evaluation 

Standards of evaluation are rooted in the 1975 formation of the Joint Committee 
on Standards for Educational Evaluation. The committee was established with the goal to 
improve the quality of educational evaluation research. The 1981 publication Standards 
for Evaluations of Educational Programs, Projects, and Materials was the committee’s 
compilation of its study. The publication was revised in 1981 and 1994 and retitled as 
Program Evaluation Standards (Davidson, 2005; Fitzpatrick, Sanders, & Worthen, 2004; 
Gall, Gall, & Borg, 2003). 

Stakeholders are individuals affected by the results of an evaluative study. The 

potential effect on stakeholders mandates that a study be of a quality that contributes to 

the institution’s sensitivity to the changing needs of those stakeholders (McGillin, 2003). 

Charles and Mertler (2002) wrote, “Evaluation research is done to determine the 

relative merits of various products and approaches used in education” (p. 3 1 1). Airasian 

and Gay (2003) explained, “Evaluation research is concerned with making decisions 

about the quality, effectiveness, merit, or value of educational programs, products, or 

practice” (p. 7). In a discussion of criteria for a good evaluation study, Gall et al. (2003) 

described the purpose of an evaluation: 

An evaluation has utility if it is informative, timely, and useful to the affected 
persons. Feasibility means, first, that the evaluation design is appropriate to the 
setting in which the study is to be conducted and second, that the design is 
cost-effective. Finally, accuracy refers to the extent to which an evaluation study 
has produced valid, reliable, and comprehensive information for making 
judgments of a program’s worth, (p. 552) 

The assurance that a study is feasible, proper and accurate will provide guidance 
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in developing appropriate evaluation practices. Gall et al. (2003) discussed how the 
criteria of utility, feasibility, propriety, and accuracy are linked to the standards of 
evaluation. Multiple criteria are associated with each of the four standards. The complete 
linkage between the criteria and the standards of evaluation was summarized by the Joint 
Committee on Standards for Educational Evaluation Programs (1994). The detailed 
linkage between the standards and the criteria are included Appendix G. 

The use of multiple sources of data collection will improve the accuracy of the 
study (Fitzpatrick et al., 2004; Thomas, 2000). Multiple sources may include the use of 
literature reviews, interviews, and surveys. 

Varcoe (2003) wrote that components of an evaluation process are agreement on 
criteria, determination of gaps between the gathered data and the identified outcomes of 
the studied process, and the utilization of gap analysis to identify weaknesses in the 
process or provide feedback to identify potential changes. According to Varcoe, an 
effective evaluation study utili z es the action research methodology. Varcoe identified 
steps involved in the methodology: 

1. Define the problem and describe the purpose. 

2. Establish the research questions. 

3. Complete information gathering. 

4. Establish criteria. 

5. Develop assessment design. 

6. Implement assessment processes. 

7. Analyze and report data. 

8. Conduct gap analysis to determine findings. 

9. Use the findings as the basis for recommendations and decisions, (p. 8) 

The literature review identified two evaluation methodologies (Gall et al., 2003; 

McMillan & Schumacher, 2001). The methodologies were the context, input, process, 
and product (CIPP) model and the objectives model. The CIPP methodology developed 
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by Stufflebeam (as cited in Gall et al., 2003) evaluates existing programs or programs 
under development. The CIPP method consists of four procedures. The first stage, 
context, identifies problems in the program being evaluated. The second stage, input, 
judges what resources and strategies are required to fulfill program goals and objectives. 
The third stage, process, is the collection of data. A requirement of this stage is that the 
program is in operation or the design is completed. The fourth stage, product, evaluates 
the product or program. This evaluation examines whether or not program goals were 
achieved. The CIPP model is complex and difficult to use in its entirety in a thesis or 
dissertation. 

The second methodology identified was the objectives model. The focus of this 
methodology is to identify specific goals and objectives and evaluate whether or not they 
were achieved. The objectives method measures outcomes of a program (McMillan & 
Schumacher, 2001). The objectives model is similar to that of Varcoe (2003) in that the 
researcher identifies discrepancies, or gaps, in a program (Gall et al., 2003; McMillan & 
Schumacher). Gall et al. also wrote that the objectives model is used to evaluate the costs 
and benefits of a program. A cost-benefit analysis is dependent upon whether the costs 
and benefits are calculable in monetary terms. 

Survey Instruments 

Program evaluations frequently include surveys as a source of information 
collection. The survey instrument is an important tool that allows researchers to combine 
and summarize data in evaluation studies (Fitzpatrick et al., 2004; Stufflebeam, 2001). A 
review of the literature associated with surveys indicated that surveys examine limited 
issues through the dissemination of questions to a large group of participants (Gall et al., 
2003; Fitzpatrick et al.). Many surveys use a rating scale or Likert-like items (Passmore, 
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Dobbie, Parchman, & Tysinger, 2002). Likert-like questions evaluate participant replies 
in an ordered continuum of responses from strongly agree to strongly disagree. 
Researchers found that the number of responses used in a Likert survey varies from 3 to 
10, but 5 is considered acceptable (Amoo & Friedman, 2001; Passmore et al.). The 
resulting participant perceptions allow for the interpretation of a ranking order or 
suggestion of a pattern. A survey, however, may not be an accurate reflection of the true 
importance of the interpreted data. 

Researchers have suggested that surveys possess limitations as research tools. The 

limitations appear to be rooted in three assumptions. First, surveys rely on the honesty of 

the respondents. Second, the accuracy of the data collected is dependent on how well the 

survey participants understand the survey questions. Participant education and cultural 

background can influence the validity of the responses. Third, the number of surveys 

returned may limit the value of a survey (Passmore et al., 2002; Pike & Kuh, 2005). 

The survey limitations relate to the reliability of participant responses. Survey 

reliability is dependent upon the basic assumption that the participants will candidly 

complete the survey. Pike and Kuh (2005) addressed the issue of inaccurate self-reports: 

Research has shown that self-reports are likely to be valid under five conditions: 
(1) the information requested is known to the respondent; (2) the questions are 
phrased clearly and unambiguously; (3) the questions refer to recent activities; (4) 
the respondents think the questions merit a serious and thoughtful response; (5) 
answering the questions does not threaten, embarrass, or violate the privacy of the 
respondent or encourage the respondent to respond in socially desirable ways. (p. 
282) 

Charles and Mertler (2002) also commented on survey reliability and first 
assumed that reliability may be related to survey construction. The second reliability 
assumption was based upon the premise that the survey would prompt a process that 
would “elicit and probe responses from participants and informants” (p. 162). 
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A third limitation of a survey focused on the number of returned surveys. 
Passmore et al. (2002) noted that, to achieve an acceptable rate of return, a follow-up 
protocol for nonresponding subjects is necessary. The researchers considered a survey 
response rate of around 60% to be adequate. 

Reliability may be increased if the survey is constructed through a methodical 
process. Passmore et al. (2002) put forward a process that would contribute to a reliable 
survey: 

1. Stating the problem and planning the project. 

2. Stating the research question. 

3. Reviewing the literature. 

4. Developing survey items. 

5. Constructing the survey. 

6. Piloting the survey. 

7. Administering the survey. 

Evaluation Reports 

Evaluation studies identify gaps between observed results and original criteria in 
an existing process. The gap analysis results facilitate the creation of recommendations 
for practices development and process improvement within a program. To provide 
accurate information, the evaluation report is an essential component in communicating 
the results of an evaluation study. Wurzbach (2002) identified that a properly authored 
evaluation report provides the following resources: 

1. A critical analysis of the evaluation results. 

2. A substantive product for the institution. 

3. Evidence of the effectiveness the carefully developed process. 
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4. A record of activities for the guidance of future process planning. 

5. Assistance to others in the development of similar processes. 

6. A dependable foundation for future program evaluations. 

There are five additional key concepts of an evaluation report: (a) incorporation of 
appropriate visual representations of results, (b) a clear and concise manner of writing, 

(c) use of headings and subheadings to generate a valuable table of contents, (d) 
preparation of an accurate and succinct executive summary, and (e) solicitation of 
feedback on report drafts before submission of the final report (McDavid & Hawthorn, 
2006; Wurzbach, 2002). 

Presentation of final evaluation reports may vary in format due to the variety of 
content and the differing interests of stakeholders. Individual institutional preferences 
range from written reports to oral presentations. An evaluation report is ultimately 
considered effective so long as the report provides appropriate feedback and contributes 
value to the existing service, product, or process (Fitzpatrick et al., 2004). The report 
design and content is determined by both the audience and the intended use of the 
evaluation. The value of an evaluation report lies in the ability of the writer to effectively 
communicate the contents, regardless of the of delivery mode. Although the evaluation 
study sponsor will direct the report format, an evaluation report outline will include (a) 
the purpose of the process or program, (b) identification of what was evaluated, (c) the 
purpose for conducting the evaluation, (d) the evaluation methodology, (e) impediments 
the researcher encountered conducting the evaluation, and (f) a discussion of the results 
(Wurzbach, 2002). 

Conclusion 



The review of the literature indicated that student engagement impacts the ability 
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of students to learn. Researchers discussed the historical perspective of student 
engagement. A variety of engagement activities has been identified through this review. 
Learning engagement activities are linked to criminal justice programs by several 
authors. These various researchers suggested that the involvement of student in 
experiential learning or service learning adds a beneficial component to the criminal 
justice curriculum. 

Student engagement is not a singular issue. It is influenced by instructors, 
instructional practice, and learning communities. To maximize student engagement all 
factors, students, faculty, instructional practice, institutional norms and learning 
communities are obliged to coordinate efforts to effectuate best practices. 
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Chapter 3: Methodology and Procedures 

Methodology 

This study utilized the evaluation research methodology in an action research 
study to evaluate student engagement at a Midwestern technical college’s criminal justice 
program. Charles and Mertler (2002) asserted, “Evaluation research is done to determine 
the relative merits of various products and approaches used in education” (p. 3 1 1). This 
study attempted to identify current engagement practices and any gap between current 
practices and the expected outcomes of best scenarios and to identify concepts that would 
support a recommended engagement process. Airasian and Gay (2003) supplied 
reinforcement for using the evaluative method as they described the purpose of 
evaluation research: “Evaluation research is concerned with making decisions about the 
quality, effectiveness, merit, or value of educational programs, products, or practice” (p. 
7). 

The study utilized an action evaluation methodology. This method was identified 
after reviewing the CIPP evaluation method (Gall et al., 2003) and the objectives-oriented 
evaluation approaches (Fitzpatrick et al., 2004). The CIPP model was eliminated because 
it focused on an evaluation that would contribute to management decision making 
regarding existing programs to provide an insight into future program improvements. 

The Tyler objectives-oriented approach is a research method designed to evaluate 
whether program objectives are being met (Fitzpatrick et al., 2004). The focus of this 
study was what engagement practices were used to enable students to meet course and 
program objectives. The process focus of this project made the objectives-oriented 
method incompatible with the study. One assumption of this study was that course and 
program objectives were currently being satisfied. For the aforementioned reasons, the 
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Tyler objectives-oriented approach was eliminated from consideration. 

The data-collection process used a survey tool administered to participants for 
self-reporting of information. Charles and Mertler (2002) said that one assumption 
associated with self-reporting is that the participants will honestly report. The assumption 
is based upon the premise that the survey will prompt a process that will “elicit and probe 
responses from participants and informants” (p. 162). 

Pike and Kuh (2005) addressed the issue of inaccurate self-reporting and wrote 
that a well- written survey will negate inaccurate responses. They identified five 
conditions that will enhance survey accuracy: 

1. Ask questions to which participants know, or are reasonably expected to know, 
the answer. 

2. Phrase the questions in a manner that is clear and explicit. 

3. Refer to recent occurrences. 

4. Elicit an insightful response from the participants. 

5. Respect the participants in order to elicit honest answers. 

Procedures 

Twelve procedures were proposed to address the five research questions in this 
applied dissertation study. The study included an extensive literature review that 
incorporated current and seminal publications and the development of an evaluation 
instrument that satisfied the college’s institutional review process. The literature review 
and development of the evaluation instrument addressed Research Question 1, What 
current engagement activities are available for criminal justice students at the college? 

Second, the study examined engagement activities currently in use to determine 
the expected outcomes of engagement activities for criminal justice students. This 
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examination addressed what current course engagement activities were available to 
criminal justice students (see Appendix H). An introductory letter explaining the survey 
and inviting students to participate accompanied the survey (see Appendix I). 

Third, a formative committee was established. The committee consisted of three 
members. The members were stakeholders internal to the college and associated with the 
criminal justice department. The stakeholders possessed an interest in the success of 
students and the program. The purpose of this committee was to provide input during the 
establishment of criteria, the development of an interview tool, the development of an 
evaluation instrument, the evaluation criteria, and the review of findings for the report. 

Based on their experience, the members of the formative committee were 
expected to provide suggestions related to the study. Committee members included an 
adjunct instructor, the criminal justice academic counselor, and a full-time criminal 
justice instructor. Formal documentation of the committee members and their 
qualifications is included as see Appendix J. Four issues were shared with the formative 
committee: (a) the expected outcomes of the current student engagement practices that 
were designated on the overview document, (b) the expected outcomes for best practices 
of related topics identified through the literature review, (c) the survey tool used to 
collect data from students regarding engagement practices, and (d) the survey tool used to 
collect data from criminal justice instructors regarding engagement practices. 

Formal invitations to serve on the formative committee were sent through letters 
of invitation to all committee members. Accompanying the invitation was a participation 
agreement form. All committee participants signed the participation agreement. The 
formative committee members met in face-to-face sessions or via electronic or telephonic 
modalities. An iterative process was utilized when working with the formative 
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committee. The iterative process was a process used by the committee members to review 
and examine criteria from the overview document, related topics and models from the 
literature review, and survey tools to collect data. The formative committee members 
were provided a feedback form on which to record comments, and the researcher 
requested feedback within 5 days. The researcher addressed responses from the formative 
committee members until consensus was attained that the most complete list of criteria of 
current practices and best practices were developed. The study outcomes, survey 
instrument, and evaluation criteria were deemed appropriate after no more comments 
were received. The formative committee addressed Research Questions 1, 2, 3, 4, and 5: 

1. What current engagement activities are available for criminal justice students at 
the college? 

2. What models are identified in the literature of expected outcomes for best 
practices of successful student engagement processes and programs? 

3. What are the perceptions of students and instructors regarding student 
engagement at the college? 

4. What valid components are necessary to formulate an evaluation tool that will 
assess the worth of the engagement process currently offered to criminal justice students? 

5. What would be included in a valid plan to revise, implement and evaluate an 
improved academic engagement process? 

Fourth, a summative committee was formed. The committee consisted of three 
members. The members of this committee were external to the college. Members were 
experienced in the technical college system and possessed expertise in the education 
field. Members of the summative committee included (a) a dean of a technical college 
criminal justice program, (b) a retired vice president of instruction of a technical college 
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who serves in the same capacity at a private 4- year college, and (c) a retired social 
science instructor and criminal justice manager (see Appendix J). Formal invitations to 
serve on the summative committee were sent through letters of invitation to all committee 
members. Accompanying the invitation was a participation agreement form. All 
committee participants signed the participation agreement form. The summative 
committee met in face-to-face sessions or via electronic or telephonic modalities. An 
iterative process was utilized when working with the summative committee. The iterative 
process was a process that the committee members used to review and examine criteria 
from the overview document and continued to address comments. The researcher 
presented all current practices and expected outcomes (criteria) to the members of the 
summative committee. The summative committee members were provided a feedback 
form on which to record comments, and the researcher requested feedback be returned 
within 5 days. The researcher addressed responses from summative committee members 
until consensus was attained that the most complete list of criteria of current practices and 
best practices were developed. Current practices and best practices criteria were deemed 
valid after no more comments were received. 

The summative committee addressed Research Question 3, What are the 
perceptions of students and instructors regarding student engagement at the college? and 
Research Question 5, What would be included in a valid plan to revise, implement, and 
evaluate an improved academic engagement process? 

Valid and reliable survey tools were necessary to obtain data from students who 
were currently taking criminal justice courses at the college. The literature review 
identified two previously administered survey tools. The first was the CCSSE. This 
survey focused specifically on student engagement data associated with 2-year colleges. 
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The college participated in this survey in 2004. Permission was obtained to use the results 
of the 2004 survey for this study. 

The second survey instrument was the NSSE. The focus of the survey was first 
semester freshmen and graduating seniors at 4-year colleges. Both the CCSSE and NSSE 
surveys possessed similarities. The researcher consulted both the CCSSE and NSSE as 
models in developing two surveys. Both surveys were considered models, as both 
possessed similarities. The first was a survey intended to identify the engagement 
practices in which students participated (see Appendix H). The second survey identified 
faculty knowledge and understanding of existing student engagement practices (see 
Appendix K). 

The survey tools were built upon on the research questions, outcomes pertinent to 
student engagement practices used by the criminal justice department at the college, and 
synthesis of the literature review. The director of research at the college reviewed the 
survey. Construct validity and reliability of the survey tools were established using two 
processes. The survey tools were submitted to the director of research at the college as 
part of the college institutional review procedure. These tools were approved for use in 
this study. The formative committee also reviewed the survey tools for reliability and 
readability. The surveys were also submitted to the summative committee for validation. 
Survey validity was determined by showing that survey questions addressed specific 
research questions (Gall et al., 2003). 

The survey tools and the cover letter were reviewed for reliability by checking for 
ambiguous questions or poor readability. An additional review process used the Flesch 
reading ease and the Flesch-Kincaid grade-level scales included in the spelling and 
grammar check integrated into Microsoft Word. 
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The student survey was administered during a class period to students who agreed 
to participate. Participants received a printed copy of the survey participation invitation 
(see Appendix I). To ensure confidentiality, the names of participants were not requested. 
The completed surveys were filed and then were destroyed upon completion of the 
research. 

A second survey titled Faculty Survey targeted criminal justice faculty knowledge 
and understanding of student engagement (see Appendix K). Distribution of the survey 
was made to all full-time faculty members and selected adjunct instructors who were 
active primary course instructors. Five full-time instructors and ten primary adjunct 
instructors were asked to participate in the survey. An introductory cover letter explained 
the purpose of the survey (see Appendix I). The survey and the letter were reviewed for 
reliability by checking for ambiguous questions or poor readability. The review process 
included a review by the formative committee. The surveys also were analyzed using the 
Flesch reading ease and the Flesch-Kincaid grade-level scales included in spelling and 
grammar check integrated with Microsoft Word. The researcher contacted all criminal 
justice faculty via e-mail to request participation. 

Survey participants received an electronic copy of the survey participation 
invitation (see Appendix I). Once participants agreed to complete the survey, the survey 
was personally delivered to full-time faculty and sent electronically to adjunct faculty 
participants. Names of participants were not used in order to ensure confidentiality. The 
completed surveys were filed and then destroyed upon completion of the research. 

The survey instruments contained questions and statements based on criteria 
developed by the researcher. Additional input received from the formative committee and 
summative committee through the iterative review processes were considered. Survey 
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tools were refined based on formative and summative committee comments as 
appropriate. The researcher addressed formative and summative committee comments 
until all comments were addressed and the survey tools were determined to be valid. 
However, based on the literature review, a possibility existed that the student and faculty 
survey tools would closely resemble the national survey tools. This validation process 
utilizing the formative and summative committees addressed Research Questions 1, 2, 3, 
4, and 5: 

1. What current engagement activities are available for criminal justice students at 
the college? 

2. What models are identified in the literature of expected outcomes for best 
practices of successful student engagement processes and programs? 

3. What are the perceptions of students and instructors regarding student 
engagement at the college? 

4. What valid components are necessary to formulate an evaluation tool that will 
assess the worth of the engagement process currently offered to criminal justice students? 

5. What would be included in a valid plan to revise, implement, and evaluate an 
improved academic engagement process? 

Fifth, the entire composite of expected outcomes for recommendations for 
improving student engagement, including criteria from the related issues found in the 
literature review, models of student engagement practices discovered in the literature 
review, and results of the preliminary survey tools, were presented to the formative and 
summative committees. The formative committee worked with the researcher to deem the 
criteria of best practices most appropriate for the study. The summative committee 
validated the entire composite of expected outcomes for best practices. The iterative 
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process was employed by the formative and summative committee members until all 
comments were addressed. Committee members reviewed and examined the criteria and 
presented comments to the researcher, and comments continued to be addressed until no 
additional comments were made. At that time, criteria were deemed valid. The researcher 
requested feedback forms to be completed and returned within 5 days. 

Sixth, the researcher designed the evaluation instrument. This evaluation 
instrument was composed of the following three sections: (a) expected outcomes of 
current of student engagement practices used by the criminal justice department, (b) 
expected outcomes of best possible practices for student engagement practices, and (c) a 
gap analysis section to indicate possible gaps between the current practices and best 
possible practices for the student engagement process. Recommendations were offered 
for resolving each of the gaps that were found in the evaluation. The sixth procedure 
addressed Research Question 4, What valid components are necessary to formulate an 
evaluation tool that will assess the worth of the engagement process currently offered to 
criminal justice students? and Research Question 5, What would be included in a valid 
plan to revise, implement and evaluate an improved academic engagement process? 

Seventh, the evaluation tool was presented to the formative committee for review 
and examination. The formative committee members reviewed and examined the 
evaluation tool and presented comments to the researcher. The researcher requested 
feedback forms to be completed and returned within 5 days. Comments continued until 
all were addressed. The seventh procedure addressed Research Question 3, What are the 
perceptions of students and instructors regarding student engagement at the college? 

Eighth, the evaluation tool was presented to the summative committee for 
validation. The members reviewed and examined the evaluation tool and presented 
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comments to the researcher. The researcher requested feedback forms to be completed 
and returned within 5 days. Comments continued until all comments were addressed. The 
evaluation tool was deemed valid when the summative committee no longer provided 
comments. 

Ninth, an expert evaluation panel was formed. This panel consisted of two experts 
in the field of evaluation who were external to the college. The members of the 
evaluation panel were identified in the final report. The members of this expert evaluation 
panel conducted the evaluation by comparing each expected outcome (criterion) of 
recommendation to each expected outcome (criterion) of current practices of student 
engagement practices employed by the criminal justice department. The researcher was 
available during the evaluation process to answer any questions from the expert 
evaluation panel regarding any of the criteria on the evaluation tool, and the expert 
evaluation panel compiled an evaluation report that listed any gaps between current 
practices and best practices along with a recommendation for each gap that was found. 

Tenth, the evaluation report was submitted to the formative committee and the 
summative committee for review, comments, and validation. Once again, an iterative 
process was employed. Committee members reviewed and examined the evaluation 
report and presented comments to the researcher. The researcher requested feedback 
forms to be completed and returned within 5 days. Comments continued until all were 
addressed. At that time, the evaluation report was considered valid. This procedure 
addressed Research Question 5, What would be included in a valid plan to revise, 
implement and evaluate an improved academic engagement process? 

Eleventh, the evaluation report was submitted to the summative committee for 
validation. Once again, an iterative process was employed. Committee members 
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reviewed and examined the evaluation report and presented comments to the researcher. 
The researcher requested feedback forms to be completed and returned within 5 days. 
Comments continued until all comments were addressed. The evaluation report was 
considered valid at that time. This procedure addressed Research Question 5, What would 
be included in a valid plan to revise, implement and evaluate an improved academic 
engagement process? 

Twelfth, the evaluation report was submitted to the executive assistant to the 
president/chief academic officer and the criminal justice dean. The researcher was 
available to meet with the executive assistant to the president/chief academic officer and 
the criminal justice dean to discuss results and address feedback. Results of the final 
report were used to determine whether changes to improve student engagement practices 
in the criminal justice department were indicated. The executive assistant to the 
president/chief academic officer and the criminal justice dean determined how the 
feedback was to be utilized. The dissemination protocol was described in the final report 
(see Appendix L). 

Proposed Timeline 

The time to conduct this study was approximately 8 months. During this time, the 
researcher gathered, analyzed, and recorded the data and results of the evaluation of the 
student engagement practices used in the criminal justice department at the college. This 
timeline allowed sufficient time for formative and summative committee feedback, 
distribution of surveys, and the report preparation. 

Assumptions 

Several assumptions were pertinent to this study. First, it was assumed that the 
reviewed literature exhausted the available information on the engagement topic, and that 
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the information contained in the literature was accurate. Second, it was assumed that the 
members of the formative, summative and expert committees utilized their knowledge to 
the best of their abilities to provide input and validation to the study. Third, the writer 
assumed that the students participating in the survey study were representative of the 
general criminal justice department student population. Fourth, the writer assumed the 
college personnel being surveyed were knowledgeable and forthright regarding their 
answers to the survey questions. Fifth, the writer assumed the surveys were reliable and 
valid as stated by feedback from the formative and summative committees. Sixth, the 
writer assumed that the collection and interpretation of the data from the surveys was 
accurate. 

Limitations 

In discussing limitations, Charles and Mertler (2000) explained, “Limitations 
refer to conditions outside the investigator’s control that affect data collection” (p. 71). 
Several limitations for this study were identified. First, varying degrees of knowledge and 
expertise among the formative and summative committee members existed. Second, the 
motivations, commitments, and expectations of the formative and summative committee 
members impacted the direction of the research study. Third, because the study focused 
on one department at a 2-year technical college, the results may not be generalized to 
other educational institutions or other divisions or departments within the college. Fourth, 
not all participants responded to the survey. 

Implications for the Improvement of Educational Practice 

Three outcomes of this study contribute to improving educational practice. First, 
the study provides the college with additional resources to facilitate criminal justice 
student learning. Second, this research project contributes to the body of research 
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focusing on student engagement in 2-year colleges. Third, the results of this study 
identify any gap in the use of current student engagement practices presently used by 
criminal justice faculty and the practices identified through the literature review. 
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Chapter 4: Results 

The purpose of this applied dissertation study was to evaluate student engagement 
practices at a Midwestern technical college criminal justice department to determine if 
the current practices were effective. The evaluation methodology was employed in 
developing an evaluation tool to conduct the study. Although several possible evaluation 
models were considered, the Varcoe (2003) design was identified as the most appropriate 
to determine if any gaps existed between current engagement practices and student 
engagement practices identified in the literature review. 

Research Question 1 

The first research question asked, What current engagement activities are 
available for criminal justice students at the college? The following procedures were 
conducted in order to answer this question. First, a review of literature in relation to 
criteria development was conducted. Criteria are described as the ideal state of a 
program, activity, product, or process. Criteria identify those portions of the activities that 
are good or of value (Fitzpatrick et al., 2004; Varcoe, 2003). An evaluation study depends 
on the criteria of the current program or process to provide a comparison of ideal criteria 
to the existing practice or process. Evaluative studies such as this one direct attention to 
gaps between current practices or processes and the ideal criteria. The identified gaps 
allow the researcher to make judgments and recommendations regarding the practices or 
processes evaluated. The recommendations form the foundation for improvement of the 
quality of the practice or process. No formal engagement practices or processes are 
currently identified within the criminal justice department. 

Existing practices were identified through the use of two surveys (see Appendixes 
H and K). These surveys were validated during the Midwestern technical college 
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institutional review process. Validation of the surveys through the institutional review 
process was required prior to administration of the surveys. 

Second, a formative committee was established to provide input to the study. Four 
issues were shared with the formative committee: (a) the expected criteria of the current 
student engagement practices that were designated in the evaluation tool document, (b) 
the expected criteria of recommended practices of related topics identified through the 
literature review, (c) the survey tool utilized to collect data from students regarding 
engagement practices, and (d) the survey tool utilized to collect data from criminal justice 
instructors regarding engagement practices. 

The formative committee was also utilized to review and provide input regarding 
the development of the evaluation tool and the final evaluation report. Formative 
committee members were internal to the college and associated with the criminal justice 
department. Each member possessed appreciable experience and knowledge, as well as a 
stake in the success of the criminal justice program at the college. The formative 
committee provided valuable input and quality information for this study. Committee 
members included the following persons: (a) an adjunct instructor, (b) the criminal justice 
academic counselor, and (c) a full-time criminal justice instructor. The qualifications of 
the formative committee members are included in Appendix J. Each formative committee 
member was invited by the researcher to participate in this study. All committee members 
were provided a letter of invitation, and a signed committee participation agreement form 
was returned. 

An iterative process was utilized. This process allowed the committee members to 
review submitted materials and provide feedback to the researcher. For example, the 
committee members reviewed and examined criteria provided and considered related 
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topics and models from the literature review. The data collection tools approved by the 
college were shared with the committee, along with the evaluation tool and evaluation 
report. 

Third, the researcher presented the initial criteria document that set forth the 
criteria to the formative committee for review and recommendations. The researcher 
e-mailed the document to each committee member. The e-mail contained a draft of the 
criteria document, an informational memorandum, and a feedback form. Committee 
members were asked to review the materials and return the completed feedback form 
within 5 days. 

The researcher personally met with, and received recommendations from, 
formative committee members. These recommendations prompted revisions related to 
developing criteria for the evaluation tool. Committee feedback included several 
suggestions for consideration. The first suggestion identified a cross-reference between 
the words criteria and outcomes. One member suggested that these two terms should not 
be utilized as analogous terms, as such terms represent conceptual differences. 

A second suggestion was not to identify standards. This committee member 
postulated that defining standards at that juncture in the study was premature and could 
lead to confusion. The third suggestion was to change the criteria from a question format 
to a statement format and that the criteria statements should begin with action verbs. A 
fourth suggestion was that criteria were linked to specific research questions in a table 
format. One committee member opined that this linkage suggested the study outcomes 
were predetermined and elimination of that linkage removed any appearance of bias in 
the study. This suggestion resulted in the elimination of the appendix containing the 
table. A fifth observation suggested that the criteria should be included in the design of 
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the evaluation tool. Another committee member suggested that inclusion of the criteria in 
evaluation tool this format would be more understandable and would provide the 
opportunity to facilitate comments regarding the evaluation tool and potentially expedite 
the work of the committee. 

A committee member expressed a preference to meet individually and discuss the 
study criteria. A meeting was scheduled, and two of the three formative committee 
members attended. Prior to the meeting, the suggested changes from the committee were 
drafted and shared with each committee member via e-mail. The attending members 
accepted all changes following a discussion of the changes and the evaluation tool. The 
absent member replied by e-mail and stated that the revised criteria and evaluation tool 
were acceptable. 

Fourth, a summative committee was formed. The committee consisted of three 
members. All members of the summative committee were external to the college. 
Members possessed experience with the technical college system and possessed expertise 
in the education field. Summative committee members included the dean of a technical 
college criminal justice program, a retired vice president of instruction of a technical 
college currently serving in the same capacity at a private 4-year college, and a retired 
criminal justice manager and social science instructor (see Appendix J). 

Formal invitations to serve on the summative committee were sent through letters 
of invitation to all committee members. A participation agreement form accompanied the 
invitation. All committee participants signed and returned the participation agreement 
form. The summative committee met in face-to-face sessions and via electronic or 
telephonic modalities. An iterative process was utilized when working with the 
summative committee, and this process was also utilized to review the study criteria and 
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the evaluation tool. The researcher presented all current practices and expected criteria to 
the members of the summative committee. The summative committee members were 
provided a feedback form on which to record their comments. The researcher requested 
feedback within 5 days. 

Committee members responded to the evaluation tool and the proposed criteria. 
Aside from several grammatical suggestions, members accepted the criteria and 
evaluation tool. The validated criteria and evaluation instrument are presented in 
Appendix M. 

Research Question 2 

The second research question asked, What models are identified in the literature 
of expected outcomes for best practices of successful student engagement processes and 
programs? Literature relevant to student engagement was reviewed to answer this 
research question. The literature review was assembled to obtain a comprehensive sample 
of applicable scholarly information as well as models to compare with the existing 
student engagement practices in the criminal justice department at the college. More than 
90 sources were consulted. The research consisted of the works of a variety of authors 
and institutional documents from colleges as well as other academic works. Research 
included journal articles, textbooks, monographs, and dissertations. Most sources were 
derived from research within the past 10 years, with the exception of landmark and 
historical research. 

The literature review presented a compilation of preferred criteria for 
development of effective student engagement practices and provided a review of a 
number of student engagement practices that incorporate such elements as faculty 
involvement, andragogy, learning communities, and experiential learning. The criteria 
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represented best practices of student engagement at various colleges and universities. The 
practices were utilized to develop the criteria for Section 2 of the evaluation tool. 
Research Question 3 

The third research question asked. What are the perceptions of students and 
instructors regarding student engagement at the college? Several procedures were utilized 
to address this question. The first procedure included the literature review that identified 
appropriate information related to student engagement practices and existing survey 
tools. 

The second procedure involved the development of the survey instruments. The 
survey tools were developed utilizing the study research questions and a synthesis of the 
literature review and were submitted to the director of research at the college as required 
by the college institutional review procedure. Construct validity and reliability of the 
survey tools were established using two processes. The college requires that any research 
be approved through the local institutional review process, and this study was submitted 
for the local review. The college’s institutional review resulted in approval to conduct the 
study. The review board directed that the reviewed surveys be utilized, and the college 
directive was honored. The surveys were also provided to both the formative and 
summative committees. 

Program evaluations frequently utilize surveys as one source of information 
collection. The survey instrument is an important tool that allows researchers to combine 
and summarize data in evaluation studies (Fitzpatrick et al., 2004; Stufflebeam, 2001). A 
review of the literature associated with surveys indicates that surveys examine limited 
issues through the dissemination of questions to a large group of participants (Gall et al., 
2003; Fitzpatrick et al.). Many surveys utilize a rating scale or Likert items (Passmore et 
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al., 2002). Likert questions evaluate participant replies in an ordered continuum of 
responses from strongly agree to strongly disagree. Research indicated that the number 
of responses utilized in a Likert survey varies from 3 to 10, but 5 is considered acceptable 
(Amoo & Friedman, 2001; Passmore et al.). The resulting participant perceptions allow 
for the interpretation of a ranking order or the suggestion of a pattern. A survey, however, 
may not be an accurate reflection of the true importance of the data. 

Surveys possess limitations as research tools. The limitations are rooted in three 
assumptions: first, the honesty of the respondents; second, how well the survey 
participants understand the survey questions; and third, the survey response rate 
(Passmore et al., 2002; Pike & Kuh, 2005). Effective surveys utilize a systematic 
development process. The process includes stating the problem, planning the project, 
stating the research question, reviewing the literature, developing survey items, 
constructing the survey, piloting the survey, and administering the survey (Passmore et 
al.; Pike & Kuh). 

Research supported the premise that surveys are created to meet similar reliability 
and validity standards as other methods of research. These standards are sometimes found 
to be lacking when the surveys are reviewed (Gall et al., 2003). Validity and reliability 
standards applicable to other data research collection tools also apply to surveys. 

Evidence that survey questions are responsive to stated research questions 
establishes content validity. A survey is valid on its face when the respondents and those 
who interpret the results advise that the survey appears valid, or reliable. An exception to 
this rule exists, however, and according to Gall et al. (2003), “A lower level of item 
reliability is acceptable when the data are to be analyzed and reported at the group level 
than at the level of individual respondents” (p. 223). 
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Survey questions developed for this evaluation study consisted of Likert-type 
scales and response format questions. The Likert-type scales allowed participants the 
ability to express a sense of like or dislike. This type of question provided a statement 
that was followed by responses ranging from the extremes of positive to negative. Many 
survey participants were familiar with these types of response choices. This format 
permitted the survey to be quick and effective (Passmore et al., 2000). 

Response format items were developed as either opened or closed statements or 
questions. The open response format allows subjects to express their opinions. The closed 
response questions provide the participant with a list of choices. Closed response 
statements were utilized to maximize the ease of response for participants. 

In the literature, a limited number of surveys focus on student engagement, and 
only two such surveys were identified. The first was the NSSE and the second was the 
CCSSE. The surveys were conducted to collect latent information regarding the opinions 
and perceptions of the participants (Gall et al., 2003). Survey questions were developed 
by the researcher in conjunction with the institutional review process of the Midwestern 
technical college, and the review process assisted in the validation of the survey. The 
researcher submitted two draft surveys as a partial requirement for the college 
institutional review process, and the college accepted the preliminary surveys prior to 
receiving feedback from the formative or summative committees. The process of survey 
tool development and review in conjunction with the college institutional review process 
and the expertise of the director of institutional research established the construct validity 
of the survey instruments. In addition, the formative and summative committees reviewed 
the surveys and were afforded the opportunity to provided feedback. 

Third, the researcher presented the two surveys, the student survey (see Appendix 
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H) and the faculty survey (see Appendix K), to the formative and summative committees 
for review. The documents were e-mailed to the committee members. The e-mail 
included a cover memorandum and a feedback form. Committee members were asked to 
review the materials and return the completed feedback form within 5 days. 

Fourth, the student and faculty surveys were distributed to the appropriate 
respondents. Sample size for the student survey consisted of 40 criminal justice students. 
These students were older than 1 8 years of age, included both men and women, came 
from a variety of ethnic backgrounds, and spoke English as a primary language. The 
students were randomly selected. Permission was obtained from an instructor to survey a 
class in which all students anticipated graduating at the end of the current term. 

The sample size for the faculty survey consisted of 5 full-time and 10 adjunct 
instructors in the criminal justice department. All faculty participants were older than 18 
years of age and included men and women who spoke English as a primary language. 
The entire full-time faculty was asked to participate. The adjunct faculty members were 
selected based upon those instructors actively employed during the term in which the 
survey was distributed, which was the spring term of 2007. 

The researcher explained his role, the purpose of the study, and the details of the 
participation invitation when requesting student participation in the student survey. 
Students were advised that they were not obligated to participate in the study and that the 
survey would take approximately 10 to 15 minutes to complete. Students willing to 
participate were given the participation invitation to read and review. A written 
participation letter was utilized in lieu of a consent form because the participants 
remained anonymous. The survey was distributed after the students reviewed the 
participation letter. Completed surveys were collected and filed in a locked filing cabinet 
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for the duration of the study and were destroyed upon completion of the research. 

The faculty survey was distributed personally to full-time faculty members. 
Faculty were asked to return the completed survey in an anonymous manner. The surveys 
were e-mailed to the adjunct faculty. Adjunct faculty were asked to return completed 
surveys in an anonymous manner, and completed surveys were collected and filed in a 
locked filing cabinet for the duration of the study. Surveys were destroyed upon the 
completion of the research. 

Results from the student and faculty surveys were analyzed and contributed to the 
construction of the evaluation tool. The results were compiled, with student survey results 
presented in Appendix N and faculty survey results presented in Appendix O. No 
quantitative reviews of the survey results were undertaken, as the majority of questions 
were not assigned numeric values. 

Research Question 4 

The fourth research question asked, What valid components are necessary to 
formulate an evaluation tool that will assess the worth of engagement practices currently 
offered to criminal justice students? The Varcoe (2003) research design and six 
procedures were utilized to answer Research Question 4. First, a review of the literature 
related to standards of evaluation was conducted. Current standards of evaluation were 
compiled by the Joint Committee on Standards for Educational Evaluation (1994) and 
updated in The Program Evaluation Standards: How to Assess Evaluations of 
Educational Programs (as cited in Davidson, 2005; Fitzpatrick et al., 2004; Gall et al., 
2003). 

Charles and Mertler (2002) wrote that evaluation research is concerned with 
making decisions and determining the relative merits of various programs used in 
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education. (Airasian & Gay, 2003; Charles & Mertler). Gall et al. (2003) discussed the 
criteria for a good evaluation study: 

An evaluation has utility if it is informative, timely, and useful to the affected 
persons. Feasibility means, first, that the evaluation design is appropriate to the 
setting in which the study is to be conducted and second, that the design is 
cost-effective. Finally, accuracy refers to the extent to which an evaluation study 
has produced valid, reliable, and comprehensive information for making 
judgments of a program’s worth, (p. 552) 

The assurance that a study is feasible, proper, and accurate provides guidance in 
developing appropriate evaluation instruments. Varcoe (2003) wrote that components of 
an evaluation process are agreement on criteria, determination of gaps between the 
gathered data and the identified outcomes of the studied process, and the utilization of 
gap analysis to identify weaknesses in the process or provide feedback to identify 
potential changes. Gall et al. (2003) discussed how the criteria of utility, feasibility, 
propriety, and accuracy are linked to the standards of evaluation. Multiple criteria are 
associated with each of the four standards. The complete linkage between these criteria 
and the standards of evaluation was summarized in Evaluation Standards: How to Assess 
Evaluations of Educational Programs (Joint Committee on Standards for Educational 
Evaluation Programs, 1994). The detailed linkage between the standards and the criteria 
is included as an appendix (see Appendix G). 

Second, the researcher developed a preliminary evaluation tool. A composite of 
validated evaluation tools was considered during the design of the evaluation instrument. 
The evaluation instrument was composed of three sections: (a) expected criteria of 
current student engagement practices utilized by instructors at the college, (b) expected 
criteria of engagement practices for enhancing student engagement, and (c) a gap analysis 
section to identify possible gaps between current engagement practices and practices 
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identified through the literature for student engagement, including recommendations 
focused on resolving each of the gaps identified by the evaluation. 

Third, the entire array of expected criteria related to recommended practices, 
including criteria formed as a result of the literature review and results of the survey 
tools, was shared with the formative committee. The formative committee worked with 
the researcher to determine the criteria of best practices most appropriate for the study. 
The criteria endorsed by the formative committee were forwarded to the summative 
committee who validated those criteria for recommended practices. 

Fourth, the researcher presented the evaluation tool to the formative committee 
for their review and recommendations. The evaluation tool reflected the results of the 
literature review and the faculty and student surveys. The researcher e-mailed the 
evaluation document to each committee member. The e-mail contained a draft of the 
evaluation tool, a cover memorandum, and a feedback form. Committee members were 
asked to review the materials and return the completed feedback form within 5 days. 

The formative committee members critically assessed the first draft of the 
evaluation tool and requested several grammatical changes. Committee members 
suggested all criteria identified in the evaluation tool begin with action verbs. The 
committee additionally recommended that the tool list components and activities should 
be paired with the criteria instead of merely listing several criteria. As a result, the 
researcher revised the evaluation tool to reflect the recommendations of the formative 
committee. 

An updated evaluation instrument was supplied to the committee members. 
Included with the revision was a memorandum describing how the revised evaluation 
document addressed member concerns. The formative committee accepted the tool 
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following the final review of the evaluation tool changes. 

The revised evaluation tool was forwarded to the summative committee for 
validation. The researcher conveyed the survey documents via e-mail. The e-mail 
contained the evaluation tool as revised by the formative committee, a cover 
memorandum, and a feedback form. Committee members were asked to review the 
materials and return the completed feedback form within 5 days. The researcher received 
the summative committee responses. Summative committee members accepted the 
evaluation instrument and made no additional recommendations. The evaluation tool was 
therefore considered valid. 

Fifth, an expert evaluation panel was formed consisting of two people external to 
the college who were experts in the field of evaluation (see Appendix P). The evaluation 
panel and the researcher communicated by phone, in person, and via e-mail. 

Sixth, the expert evaluation panel conducted its evaluation of the student 
engagement practices identified as being in place at the college. The panel compared each 
expected criterion of recommended practices to each expected criterion of current 
practices. The researcher communicated with the panel by telephone or e-mail during the 
evaluation process to answer any questions related to the evaluation tool. The evaluation 
panel provided its results to the researcher via e-mail (see Appendix Q). 

Research Question 5 

The fifth research question asked, What would be included in a valid 
recommendation to revise, implement and evaluate an improved academic engagement 
process? For this research question, the following six procedures were completed. 

First, a review of literature provided information on evaluation reports. According 
to Thomas (2000), inherent unreliability is associated with a single data source. 
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Evaluators neutralize this unreliability by employing multiple sources of data utilizing a 
variety of measurement tools. This evaluation study dealt with the reliability issue by 
using two surveys and a comprehensive literature review as data sources. 

Second, the evaluation report that identified gaps between current practices and 
recommended practices, including recommendations for addressing the gaps, was 
prepared. The researcher, in cooperation with the expert evaluation panel, drafted the 
report. A properly prepared evaluation report contains information focused on six areas: 
(a) a critical analysis of the evaluation results, (b) a substantive product for the institution, 
(c) evidence of the effectiveness the carefully developed process, (d) a record of activities 
for the guidance of future process planning, (e) assistance to others in the development of 
similar processes, and (f) a dependable foundation for future program evaluations 
(Wurzbach, 2002). 

Additional important concepts included in an evaluation report are: (a) the 
incorporation of appropriate visual representations of results, (b) the use of a clear and 
concise manner of writing, (c) the use of headings and subheadings to generate a valuable 
table of contents, (d) the preparation of an accurate and succinct executive summary, and 
(e) the solicitation of feedback on report drafts before submission of the final report 
(McDavid & Hawthorn, 2006; Wurzbach, 2002). 

Evaluation report format varies among researchers. No standard report format 
exists. The inability to standardize report formats is a result of the differences associated 
with the required content of each report and the diverse needs of involved stakeholders. 
The information contained in the evaluation report is fundamental to the usefulness of 
that report. Effective communication of the information enhances the value of the report 
(Fitzpatrick et al., 2004). A valuable report provides essential feedback while adding 
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value by recommending changes to an existing service, product, or process. Authors 
suggested that the design and content of a report include consideration of the audience as 
well as the intended use of the evaluation. Wurzbach (2002) suggested an evaluation 
report outline reflect the background and purpose of the evaluated process, a description 
of what was evaluated, the intended purpose of the evaluation, the methodology that was 
utilized to conduct the evaluation, obstacles encountered by the researcher while 
conducting the evaluation, and a clear discussion of the evaluation results. Incorporated 
into the evaluation report (see Appendix R) are several of these concepts. The formative 
and summative committees reviewed the report prior to finalizing the report. 

Third, the researcher wrote the evaluation report. The final version of the 
validated evaluation report is included in Appendix R. 

Fourth, the evaluation report was submitted to the formative committee for review 
and recommendations. The researcher e-mailed a draft of the evaluation report, a cover 
memorandum, and a feedback form to each committee member. Committee members 
were asked to review the materials and return the completed feedback form personally or 
via interoffice mail within 5 days. The iterative process was employed, and committee 
members reviewed and examined the evaluation report and presented comments to the 
researcher. Committee feedback was addressed until all issues were resolved. 

Committee members were provided the updated evaluation report. Included with 
the revision was a memorandum identifying how the revised document addressed 
member concerns. The formative committee accepted the evaluation report. 

Fifth, a revised draft of the evaluation report was submitted to the summative 
committee for review, comment, and final validation. The researcher provided the report 
via an e-mail that included the draft evaluation report, a cover memorandum, and a 




83 



feedback form. Committee members were asked to review the materials and return the 
completed feedback form via e-mail within 5 days. 

Recommendations from the summative committee were received, prompting the 
researcher to revise the evaluation report. Committee feedback suggested changing 
verbiage to clarify statements made in the report. A second comment suggested adding 
breadth to a recommendation. A third suggested that a recommendation was unnecessary 
and could be deleted. A fourth comment identified that an opinion was present, and 
rewording the sentence was suggested. 

The researcher addressed the concerns and sent a memorandum to the summative 
committee detailing how comments were addressed and included a revised draft of the 
evaluation report. The committee accepted the revised report. 

Sixth, the evaluation report was submitted to the dean of criminal justice and the 
executive vice president/chief academic officer of the college. The researcher met with 
the dean of criminal justice and the executive vice president/chief academic officer of the 
college individually to discuss the report results and obtain feedback. The researcher 
requested that a liaison be identified to facilitate the development and implementation of 
report recommendations found acceptable by the college. The researcher also suggested 
that the administrators report the progress of improving criminal justice student 
engagement activities to the executive cabinet. A final suggestion included establishing a 
series of periodic follow-up evaluations of student engagement to track the success of any 
implemented recommendations. The use of the CCSSE was suggested as an appropriate 
tool to reassess any changes made to the process. 

The final report results were intended to assist in the identification of changes 
necessary to improve criminal justice student engagement at the college. The criminal 
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justice dean and executive vice president/chief academic officer hold management level 
positions at the college, and in their professional capacities will determine the impact of 
the evaluation feedback upon decisions influencing policy and procedures related to 
student engagement. A dissemination protocol is included in Appendix L. 

A plan for revision, implementation, and future evaluation was prepared by the 
researcher. This implementation plan will be executed under the direction of the criminal 
justice dean and executive vice president/chief academic officer of the college as they 
deem appropriate. The administrators will assign a contact person to assist in the 
development and implementation of given assignments. As a final step, administrators 
will meet with the criminal justice faculty to review report recommendations, address 
issues raised by faculty members, and complete the implementation plan assigning 
specific duties to individual faculty. 

The implementation plan will be executed over a 6-month timeframe. Faculty 
members will meet on a monthly basis to discuss assignment progress and express 
concerns related to their assignments. Faculty assignments will be completed within the 
6-month timeframe by reporting progress in a final meeting with the criminal justice 
dean. As changes are implemented and new information is identified, the criminal justice 
dean will report the progress to the executive vice president/chief academic officer of the 
college. 

As a final step, a follow-up evaluation was suggested for the 2008-2009 academic 
year. This will provide an opportunity to implement recommended changes. The CCSSE 
was recommended as an appropriate evaluation tool. The entire plan for revision, 
implementation, and future evaluation of the revised process in 2008 is included in 



Appendix S. 
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The sequence of events culminating in the completion of the study followed a 
specific progression: 

1. In December 2006, an exhaustive review of literature was completed to identify 
recommended student engagement practices. 

2. In December 2006, formative and summative committees were organized. 
Committee members were sent e-mail invitations that included letters of participation and 
feedback forms. Members were asked to commit to assisting the researcher by signing the 
letters of participation. 

3. In January 2007, study criteria for student engagement activities were 
developed. 

4. In February 2007, the student and faculty survey tools were distributed to their 
appropriate populations. Data were gathered and reviewed in developing the evaluation 
tool. 

5. From February to March 2007, the formative and summative committees 
reviewed the composite of expected criteria for recommended practices, including criteria 
from the related issues found in the literature review, student engagement models 
identified through the literature review, and the results of the survey tools. 

6. In March 2007, the evaluation tool was designed by the researcher and 
forwarded to the evaluation panel, whose members completed the evaluation within a 
two-week period and provided results to the researcher. 

7. During May 2007, the evaluation report was written and submitted to the 
formative and summative committees for review, comments, and validation. Upon 
validation, the evaluation report was submitted to the criminal justice dean and executive 
vice president/chief academic officer. 
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Chapter 5: Discussion 

Introduction 

The purpose of the applied dissertation study was to evaluate student engagement 
practices of a Midwestern technical college criminal justice program. The evaluation 
provided guidance in identification of gaps between the engagement activities identified 
in the literature and the practices currently in use by the criminal justice faculty. 
Recommendations to improve student engagement were prepared to assist criminal 
justice faculty in closing the identified gaps. 

The results of this study of student engagement correlate with the information 
found in a broad-based literature review. Little scholarly writing focused on 2-year 
colleges. Schreiber and Shinn (2003) wrote that, whereas 44% of students seeking a 
college degree attend 2-year institutions, only a scant 5% of research focuses on 2-year 
institutions. No research specifically related to 2-year vocational institutions was located. 

The literature review identified student engagement practices utilized by various 
collegiate institutions. Astin (1985) defined student engagement as student involvement. 
Astin (1985) went on to define student involvement as referring “to the quantity and 
quality of the physical and psychological energy the student invests in the college 
experience” (p. 156). Young (2003) wrote that while engagement varies with course 
majors, the concept of engagement relates to students being involved with course 
material as well as the process of learning. 

Authors identified that engagement practices are varied and need to be designed 
carefully. Practices identified include class presentations, team-oriented assignments, and 
practical assignments focusing on solving “real world problems” (Bowden et al., 2003; 
Smith & Bath, 2006; Upcraft et al., 2005; Zhao & Kuh, 2004; Zhu & Baylen, 2005). The 
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survey instruments utilized in this study indicate that the criminal justice department 
currently utilizes the student engagement practices recognized in the literature. 

Students and faculty were asked. During this school year, how regularly have you 
done the following in your criminal justice classes? Both groups reported that, over 50% 
of the time, students participated in class activities that were engagement oriented very 
often or often. These activities included class discussions, in-class project preparation 
interacting with peers, and engaging others in discussions centered on their academic 
assignments (see Appendixes N and O). 

Students and faculty responded positively to the inquiry, During this school year, 
what emphasis has your criminal justice coursework placed on the following? The 
response categories were “use of memorization,” “concept analysis,” “finding new ways 
to integrate ideas and experiences,” “evaluating the soundness of information,” and 
becoming proficient with new skills.” Both groups again showed that emphasis was 
placed on these areas over 50% of the time as very much or quite a bit (see Appendixes N 
and O). 

The data from surveys also demonstrated inconsistencies. When asked how much 
time during a 7-day week students spent preparing for class, both students and faculty 
were again consistent. Students’ responses indicated that 75% spent fewer than 6 hours a 
week preparing for class. Faculty estimated that, 92% of the time, students spent fewer 
than 6 hours a week in activities related to preparation for class. Responding to an inquiry 
related to participation in college- sponsored activities, 95% of the students reported 
spending fewer than 6 hours a week participating in college- sponsored activities, and, of 
that number, 60% reported that they never participated in college-sponsored activities. 

Students were also asked to respond to the question. Have your skills and personal 




88 



development in the following areas been improved by the criminal justice department? 
Fifteen skills were listed as responses: (a) a broad general education, (b) job or 
work-related knowledge and skills, (c) clear and effective writing skills, (d) clear and 
effective speaking skills, (e) critical and analytical thinking, (f) numerical 
problem-solving skills, (g) information technology and computer skills, (h) ability to 
work effectively with others, (i) skills to learn effectively on your own, (j) understanding 
yourself, (k) understanding people of diverse racial/ethnic backgrounds, (1) personal code 
of values and ethics development, (m) contributing to community welfare, (n) refining 
career goals, (o) career opportunity information. Both students and faculty reported that, 
in most areas, the skills were very much or quite a bit improved over 60% of the time (see 
Appendixes N and O). 

Students reported that, less than 50% the time, skill development was improved 
very much or quite a bit. Faculty reported that skill development was improved very 
much or quite a bit in the areas of writing skills (46%) and diversity skills (33%; see 
Appendixes N and O). 

The literature review indicated that a disconnect generally existed between how 
students and faculty viewed student engagement (Astin, 1985; McClenney 2004; Weimer 
2002). Responses to the surveys correlated with those observations (see Appendixes N 
and O). In the areas of inquiry focused on student engagement activities, students and 
faculty responded similarly. Over 50% of all responses indicated that behaviors were 
performed or skills developed very often or often. However, closer examination revealed 
that, in nearly all student engagement activities analyzed, a disconnect existed between 
students and faculty responses. 

The disconnect is indicated in 17 of the 21 questions related to activities regularly 
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performed. Students rated themselves higher in 10 of the 21 areas, and faculty rated 
students higher in 7 of the 21. Only 4 of the 21 questions earned similar responses from 
both groups of participants. 

The surveys inquired into six categories related to the emphasis placed on course 
work. Faculty reported a higher emphasis on course work in four of the six inquiry areas. 
Interestingly, faculty reported a higher emphasis in areas of analysis, perspective, 
evaluation, and adaptation. All inquiry areas related to the more complex learning goals. 

An additional disconnect is present in the survey area related to how challenging 
criminal justice examinations were during the 2006-2007 school year. Using a Likert 
scale, faculty reported that examinations were more challenging than students perceived 
them. Just over half (58%) of the students reported the examinations as extremely 
difficult , whereas 77% of the faculty perceived their examinations as difficult. 

Another area the survey examined was skill development. The question was, 

Have your skills and personal development in the following areas been improved by the 
criminal justice department? Fifteen individual responses focused on cognitive and 
behavioral characteristics. Students reported improvement as very much or quite a bit at a 
higher rate than faculty in 8 of the 15 response areas. Faculty reported greater student 
improvement in one area: numeric problem-solving skills. All other response areas were 
essentially equivalent. 

Most students participating in the survey met the definition of traditional students. 
The survey indicates student characteristics as (a) 81% between the ages of 19 and 24, (b) 
75% are males, (c) 80% are single, and (d) 18% are the first generation to attend college. 

The literature review identified two additional areas contributing to increased 
student involvement. The areas are learning communities (Kellogg, 1999; Minkler, 2000; 
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Price, 2005; Smith, 2003; Webber, 2000), and service oriented learning (Bringle & 
Hatcher, 1996; Frederickson, 2000; Markus et al., 1993; Penn, 2003; Upcraft et al., 

2005). 

Learning communities group students in areas of academic commonality. These 
learning associations are calculated to integrate student’s academic and social activities. 
The integration of these activities is intended to facilitate learning centered on course 
majors or thematically linked courses (Cross, 1998; Smith & Hunter, 1988; Zhao & Kuh, 
2004). 

Service learning is another method to increase student engagement and encourage 
integration of class activities that take place outside of the traditional classroom with the 
curriculum. An extension of, or complement to, learning communities, service learning is 
designed to complement course outcome achievement (Oates & Leavitt, 2003). 
Real-world situations and service learning effectively connect students and faculty. 
Linking community service to academics allows the integration of a student’s 
interpersonal development with academic and cognitive development (Hubbert, 2002). 

Benefits attributed to service learning activities include providing students with 
the opportunity to apply newly acquired knowledge and skills (Penn, 2003) and the 
opportunity to experience academic learning in a manner that increases student interest 
and enthusiasm for their studies (Bringle & Hatcher, 1996). The benefits noted above 
lead to improved student performance. Improved performance is manifested in higher test 
scores (Markus et al., 1993). Frederickson (2000) also identified enhanced performance 
using the service learning model. He noted that students’ self-report improved 
understanding of subject matter when attending courses taught using the service-learning 



model. 
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The student survey provided an area of concern. Students reported that they spent 
little time studying. Survey Question 5 asked, How much emphasis does the criminal 
justice department place on each of the following? The student survey responses 
indicated that 45% of the respondents reported that the department places quite a bit of 
emphasis on substantial amounts of time for preparation and study. The faculty survey 
respondents concurred, reporting that faculty emphasized that substantial time studying 
was necessary. Faculty reported emphasizing preparation and study, with 54% reporting 
that they stressed studying quite a bit and 15% reporting that they stressed studying very 
much. 

Although faculty respondents placed emphasis on studying, student and faculty 
surveys indicated that students actually spent little time studying. Question 6a asked, 
During a full 7-day week, how much time, in hours, do you normally spend on each of 
the following activities? The first area of inquiry was time spent preparing for class in 
hours. Ninety-two percent of the faculty and 75% of students indicated students spent 5 
or fewer hours a week preparing for class. Question 16 asked, What is your overall grade 
average? Student survey responses indicated that 93% had a grade point average of B or 
better. 

Two factors were identified in the surveys as potential influences in the lack of 
time students devote to their studies. The first was the amount of time students spent 
working. Question 6b asked, During a full 7-day week, how much time, in hours, do you 
normally spend on each of the following activities . . . employment for pay? Fifty-five 
percent of students reported that they were employed more than 30 hours per week. A 
corresponding question to faculty was, During a full 7-day week how much time, in 
hours, do you believe your students normally spend on each of the following activities? 
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The specific area analyzed was employment for pay. Sixty-nine percent of faculty 
reported that they believed students were employed 20 hours or less per week. None of 
the faculty indicated that students worked more than 30 hours per week. 

A second factor suggested that a lack of rigor may be associated with criminal 
justice courses. The lack of rigor may be a result of a lack of substantive student 
engagement activities applied by criminal justice instructors. 

Five research questions were identified for this study. To answer these questions, 
the Varcoe (2003) research design was utilized, and an evaluation tool was developed 
utilizing the results of the literature review, engagement survey information, and 
constructive criticism from formative and summative committee members. The formative 
and summative committee members offered comments and suggestions regarding the 
criteria documents, surveys, evaluation tool, and information to be included in an 
evaluation report. Comments from committee members directed changes to study 
documents. In addition, the summative committee validated the evaluation report. An 
evaluation report was prepared in response to input of the evaluation panel. This 
evaluation process utilized the validated tool developed as part of this study. 

Additionally, a plan for dissemination of the report was developed (see Appendix L). The 
researcher provided the validated evaluation study results and recommendations to the 
dean of criminal justice and the executive vice president/chief academic officer of the 
college. The results will be shared at a scheduled meeting. The dean of criminal justice 
and the executive vice president/chief academic officer of the college will be asked to 
appraise the results and the study recommendations considering the mission of the 
college and the criminal justice division. The college’s budget will clearly be a factor that 
influences the evaluative process. The criminal justice dean will be asked to assign a 
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contact person for each of the recommendations, and contact persons will meet as a team 
with the dean to review study recommendations and prepare an implementation plan. The 
criminal justice dean will report to the executive vice president/chief academic officer of 
the college who, in turn, will inform the executive cabinet of the progress of student 
engagement changes that are made and the results of the changes as indicated. 
Conclusions 

The researcher assembled conclusions from the data obtained to address the five 
research questions. The first question asked, What current engagement activities are 
available for criminal justice students at the college? Seven criteria were identified using 
information obtained through faculty and student surveys. Formative and summative 
committee members validated these criteria: (a) involve students in cocurricular 
activities, (b) facilitate student interaction, (c) participate in activities that promote 
engagement, (d) create an environment where students and faculty interact, (e) increase 
critical thinking skills, (f) create an atmosphere for ethnically diverse information 
exchanges, and (g) increase student interaction in class activities. 

Results of the study revealed that current criminal justice department student 
engagement practices were consistent with practices identified through the literature 
review. However, the application of these practices did not achieve the goal of the 
promotion of student involvement in order to reconcile the grade point averages with the 
lack of study time reported. 

The second question asked, What models are identified in the literature of 
expected outcomes for best practices of successful student engagement processes and 
programs? The literature review focused primarily on 4-year educational institutions. A 
lack of information restricted research directly focused on 2-year colleges. The literature 
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review identified a lack of research in this area, with authors documenting that 44% of 
students attend 2-year institutions, yet only 5% of research focuses on such institutions 
(Schreiber & Shinn, 2003). 

The literature revealed a variety of engagement practices from which 
recommended engagement practice criteria were developed. These criteria were included 
in the evaluation instrument (see Appendix M). 

The literature review also identified that the concept of criminal justice student 
engagement was addressed in scholarly journals on a limited basis (Birzer, 2004; Dabney 
et al., 2006; Finckenauer, 2005; Hartmus et al., 2006; Hirschinger-Bank & Markowitz, 
2006; Kravetz, 2001; Penn, 2003; Sims, 2006). The discussions identified several 
engagement activities that enhance student learning. In addition to enhancing student 
learning, several engagement activities contributed to increasing student understanding of 
human behavior. Students were also able to identify challenges confronting the criminal 
justice system through these activities. The prevalence of these activities, throughout 
collegiate-level criminal justice education programs, cannot be determined. This inability 
may be related to a lack of scholarly writing in this area. 

The third question asked, What are the perceptions of students and instructors 
regarding student engagement at the college? The student and faculty engagement 
surveys and the literature review addressed this question. A review of the literature 
associated with surveys indicated that surveys examine limited issues through the 
dissemination of questions to a large group of participants (Fitzpatrick et al., 2004; Gall 
et al., 2003). The literature review also revealed that surveys are commonly utilized in 
evaluating practices and processes. Survey instruments are utilized to collect and 
summarize data that aids in evaluation studies (Fitzpatrick et al., 2004; Stufflebeam, 
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2001). The surveys developed by the researcher were validated through the college 
institutional review process. The formative and summative committees were consulted 
prior to distribution of the surveys. 

Two surveys were distributed to different populations. The surveys provided the 
process by which data were collected, assisting the researcher in summarizing data and 
developing an evaluation tool for the current study. The first survey, the student survey, 
collected data regarding the perceptions of criminal justice students at the college. The 
student survey participants were students in their final term. Students participating in the 
survey answered a variety of question related to learning activities, expenditure of their 
time, demographic information, and their satisfaction with the criminal justice program. 
The validated survey consisted of 23 questions. Several of the 23 questions included 
attached subquestions. 

The fourth question asked, What valid components are necessary to formulate an 
evaluation tool that will assess the worth of the engagement practices currently offered to 
criminal justice students? No evaluation instrument existed that was pertinent to this 
study, so the researcher developed an evaluation tool. The tool was developed from data 
and information collected through the administered surveys and the literature review. The 
formative and summative committees reviewed the evaluation tool. An iterative process 
was utilized to validate the tool. 

The validated evaluation tool contained 16 criteria. This tool was sent to an 
evaluation panel to perform a gap analysis. The evaluation tool provided the panel 
utilized a Likert scale for responses. The scale ranged from 1 to 5, with 5 representing 
strong agreement and 1 representing strong disagreement. The completed gap analysis is 



shown in Appendix Q. 
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The fifth question asked. What would be included in a valid recommendation to 
revise, implement and evaluate an improved academic engagement process? 
Recommendations were to be forwarded to the criminal justice dean and the executive 
vice president/chief academic officer of the college. The literature review identified 
potential criteria for inclusion in a final evaluation report (Fitzpatrick et al., 2004; 
McDavid & Hawthorn, 2006; Wurzbach, 2002). The recommendations were utilized as a 
guide for developing the report for this study. The items utilized for this report included a 
list of tables, an executive summary, a purpose for the process, a description of the 
evaluation and the purpose of the evaluation, methodology, results, appendixes, and 
references. 

Study results were categorized into two areas: (a) gaps identified through the 
evaluation tool and the gap analysis of the evaluation panel and (b) areas where the 
criminal justice program met the criteria of the validated evaluation tool. 

The criminal justice department met the criteria identified in the evaluation tool in 
nine of the identified criteria. Criteria were considered to be met if the evaluation panel 
either agreed or strongly agreed. The panel strongly agreed that the criteria evaluated 
relating to current student engagement practices were being met, and in addition, the 
panel agreed that two criteria were being satisfied. Current practices being met included, 
(a) involve students in cocurricular activities, (b) student interaction is facilitated, (c) 
course activities promote engagement, (d) an environment exists that allows for 
student-faculty interaction, (e) student thinking skills are increased, and (f) engagement 
activities are well crafted. 

The evaluation panel also disagreed with seven of the criteria. The criteria upon 
which the panel disagreed were, (a) engagement is affected by socioeconomic factors, (b) 
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importance of engagement practices is noted by faculty, (c) individual meetings occur 
between faculty and students, (d) engagement practices include past experience of 
students, (e) situational learning activities are included in courses, (f) program curriculum 
is structured to form learning communities, and (g) courses contain experiential learning 
components. 

Implications of Findings 

Three outcomes of this study contribute to improving educational practice. First, 
the study provided the college with additional resources to facilitate criminal justice 
student learning. Second, this research project contributes to the body of research 
focusing on student engagement in 2-year colleges. Third, the results of this study 
identified any gap in the use of current student engagement practices presently utilized by 
criminal justice faculty. 

Gap Analysis Results 

The evaluation indicated that opportunities for student engagement currently exist 
in the criminal justice curriculum. Students and faculty report engagement activities are 
regularly utilized. Students and faculty agree that the criminal justice division is 
supportive of students on both academic and personal levels. The evaluation panel 
indicated that the performance of the criminal justice division was deficient in seven 
areas. The deficient areas were identified through a gap analysis. Gaps identify areas for 
improvement. Recommendations for resolving the seven gaps are a result of this 
evaluation. Three of the analyzed criteria were found to be interrelated. Discussion of 
these criteria was combined. In total, five gaps were identified, and recommendations to 
address these gaps were prepared. 

The first identified gap related to student-faculty individual meetings. The college 
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identifies the importance of student-faculty interaction, as evidenced through the 
scheduling of instructor office and advising hours. Although office and advising hours 
are identified as necessary and time is allotted for this purpose, a gap continues to exist. 
Office and advising hours are scheduled by faculty based upon faculty preference. The 
scheduling of class hours is not a consideration in determining office and advising hours, 
and this may contribute to the existence of the gap. 

The second identified gap relates to capitalizing on student experiences. Most 
students in the program meet the definition of a traditional student. The definition 
identifies this student as being between the ages of 18 and 22 and being enrolled in 
full-time coursework. The presence of this gap may exist due to limited student 
experiences relative to criminal justice activities. Evaluating the scope of student 
experiences in a class is difficult. Students may be unwilling to discuss past personal 
involvement with law enforcement authorities within a classroom environment. 

The curriculum as presently constructed provides a limited number of 
opportunities to engage students in situational learning activities. Courses include 
scenario-based activities and assessments, narrowly focused on state law enforcement 
officer certification requirements and not on providing the students with engagement 
opportunities. The narrow focus limits the breadth of possibilities for situational learning. 

A third gap revealed in the evaluation is that the curriculum is not structured to 
form learning communities. The program allows students to self-select their course 
schedules. Students attend various courses together, but this is due to happenstance or 
student preference rather than by curriculum design. No coordination exists between 
criminal justice courses and applicable required general education courses. 

A fourth gap is the importance faculty place on student engagement. The degree 
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of importance a particular faculty member places on engagement related activities is 
difficult to measure. The panel indicated that faculty do not place importance on student 
engagement activities. 

The fifth gap identified is whether the diversity of students’ socioeconomic 
conditions is addressed by division student engagement practices. The student survey and 
the literature review both support the existence of this gap. Students who attend 2-year 
institutions maintain employment during the course of their studies. The student survey 
indicates that 55% of the participants work more than 30 hours per week. The survey also 
identified that students receive little, if any, monetary assistance from either their parents 
or their employers. 

Limitations 

The study is limited due to the specific focus of the study: the criminal justice 
division of one college. The narrow focus of the study limits generalization of the results. 
The results of the survey research were also limited. Approximately 20% of the criminal 
justice division student body participated in the survey. 

Changes based upon certification requirements for law enforcement officers are 
ongoing, and being instituted by an agency that is not focused upon academics. 
Certification requirement changes will materially affect course structure. As these 
changes occur, the criminal justice program will adjust to incorporate such changes. The 
relevance of this evaluation study may diminish as a result of the adjustments. 
Recommendations 

Six recommendations are presented based upon results of this study. The 
commitment of the criminal justice division and the college to provide quality education 
indicates that these recommendations will be thoughtfully reviewed and implemented. 
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First, the exploration and implementation of practices designed to build learning 
communities is a beneficial undertaking for the criminal justice division. Learning 
communities will enhance student engagement and learning with the implementation of 
linked or bundled courses. 

The law enforcement profession requires practitioners to be proficient in many 
skills. In addition, professional law enforcement practitioners are expected to possess 
diversified knowledge spanning several interrelated disciplines. Linking social science 
courses with criminal justice courses will increase engagement opportunities and 
facilitate learning in both disciplines. Courses that readily lend themselves to such a 
pairing are the introduction to criminal justice course and the contemporary American 
society course. The juvenile procedures course may be linked with either introductory 
psychology or abnormal psychology, with discussions between criminal justice and social 
science faculty taking place on a regular basis to coordinate curriculum and complement 
the instructional goals of each course. 

Bundled courses are designed to create a student cohort. Through the creation of 
the cohort, students enroll in specific courses. The opportunities for engagement increase 
and learning is facilitated as the students interact frequently in different disciplines. 

Second, the necessity exists for criminal justice faculty to examine to what extent 
situational and experiential instruction methods are currently utilized, with a focus on 
what variety of situations and experiences are beneficial to the instructional process. The 
faculty will determine the design and placement of student engagement activities that will 
most benefit student learning. The implementation of this recommendation will prevent a 
duplication of activities. 

Third, improved student-faculty communication is essential. Improving the new 
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student orientation presentation may be helpful. Stressing the importance of 
student-faculty communication during the orientation is a first step toward improvement. 
The orientation provides the opportunity to advise students of what they may reasonably 
expect from faculty outside of scheduled instructional hours. 

Fourth, reexamination of the computation and scheduling of office and advising 
hours should be conducted. Office hours are computed based upon the current number of 
a faculty member’s assigned instructional hours. The number of office hours does not 
include factoring in the number of students enrolled in a course. No criteria currently 
exist to determine the number of hours a faculty advisor may schedule. 

Fifth, improvements will be made to advising services for criminal justice 
students. The current process allows students to self-advise. The current process does not 
encourage student interaction with faculty. Assigning faculty advisors to specific students 
is one action that will encourage the development of a relationship between faculty and 
students. Another action that will encourage relationships is instituting a requirement that 
a faculty advisor meet with a student a minimum of once per term. A required minimum 
number of contacts with students may also increase faculty- student communication and 
engagement. A registration hold is one option to consider in facilitating the advising 
session. 

The researcher recommends a follow-up study. This study will be conducted a 
sufficient amount of time after the recommended changes are implemented in order to 
determine the impact of the changes upon student engagement. 

The study also utilized the results of the evaluation report to create an 
implementation plan (see Appendix S). The plan will adhere to college protocols for the 
revision of curricula. The plan for revision will consist of the following procedures: (a) 
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The criminal justice dean, in consultation with the executive vice president/chief 
academic officer, will determine the recommendations appropriate for implementation; 
(b) the dean will appoint a criminal justice faculty member to assist in the development 
and implementation of tasks related to implementation; and (c) criminal justice faculty 
will meet to review the recommendations, address concerns, and develop an 
implementation plan that assigns specific duties to faculty members. 

The implementation plan will adhere to the following guidelines: (a) complete 
implementation of the revision plan within 6 months, (b) use monthly criminal justice 
faculty staff meetings to discuss progress and concerns of the implementation plan, (c) 
faculty will work cooperatively to implement the recommendations assigned by the dean, 
(d) faculty will prepare 6-month progress reports for the dean, and (e) the executive 
cabinet will receive a status report regarding the implementation plan at the discretion of 
the executive vice president/chief academic officer of the college. 

The plan for future evaluation will adhere to the following steps: (a) A follow-up 
evaluation will be conducted by the college research and development department 18 
months after the recommended changes are implemented, (b) Internet technology will be 
the method of choice for the follow-up survey, (c) students admitted to the criminal 
justice program for one academic year following plan implementation are eligible for 
participation, (d) results of the survey will be compared with this study, (e) if indicated, 
the implemented recommendations will be revised, and (f) the survey will be 
readministered 2 years following plan implementation. 
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Appendix A 

Community College Survey of Student Engagement Document 




Ill 



The Community College Student Report 



1. Did you begin college at this college or elsewhere? 

2. Thinking about this current academic term, how 


□ 


Started 

here 


□ 


Started 

elsewhere 


would you characterize your enrollment at this 
college? 

3. Have you taken this survey in another class this 


□ 


Full-time 


□ 


Less than 
full-time 


term? 


□ 


Yes 


□ 


No 



4. In your experience at this college during the current school year, about how often have you done 
each of the following? 



a. Asked questions in class or contributed to 
class discussions 

b. Made a class presentations 

c. Prepared two or more drafts of a paper or 
assignment before turning it in 

d. Worked on a paper or project that required 
integrating ideas or information from various 
sources 

e. Came to class without completing readings 
or assignments 

f. Worked with other students on projects 
during class 

g. Worked with classmates outside of class to 
prepare class assignments 

h. Tutored or taught other students (paid or 
voluntary) 

i. Participated in a community-based project 
as a part of a regular course 

j. Used the internet or instant messaging to 
work on an assignment 

k. Used e-mail to communicate with an 
instructor 

l. Discussed grades or assignments with an 
instructor 

m. Talked about career plans with an 
instructor or advisor 

n. Discussed ideas from your readings or 
classes with instructors outside of class 

o. Received prompt feedback (written or oral) 
from instructors on your performance 

p. Worked harder than you thought you could 
to meet an instructor's standards or 
expectations 

q. Worked with instructors on activities other 
than coursework 

r. Discussed ideas from your readings or 
classes with others outside of class (students, 
family members, co-workers, etc.) 



Very 




Some- 




Often 


Often 


times 


Never 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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Very 




Some- 






Often 


Often 


times 


Never 


s. Had serious conversations with students of 
a different race or ethnicity other than your 


□ 


□ 


□ 


□ 


own 










t. Had serious conversations with students 
who differ from you in terms of their 
religious beliefs, political opinions, or 
personal values 


□ 


□ 


□ 


□ 


u. Skipped class 


□ 


□ 


□ 


□ 


5. During the current school vear, to what extent has vour coursework at this college emphasized the 


following mental activities? 


Very 

much 


Quite a bit 


Some 


Very little 


a. Memorizing facts, ideas, or methods from 
your courses and readings so you can repeat 
them in pretty much the same form 

b. Analyzing the basic elements of an idea, 
experience, or theory 

c. Synthesizing and organizing ideas, 
information, or experiences in new ways 

d. Making judgments about the value or 
soundness of information, arguments, or 
methods 


□ 

□ 


□ 

□ 


□ 

□ 


□ 

□ 


□ 


□ 


□ 


□ 










□ 


□ 


□ 


□ 


e. Applying theories or concepts to practical 
problems or in new situations 


□ 


□ 


□ 


□ 


f. Using information you have read or heard 
to perform a new skill 


□ 


□ 


□ 


□ 



6. During the current school year, about how much reading and writing have you done at this college ? 





None 


1 to 4 


5 to 10 


1 1 to 20 


More than 20 


a. Number of assigned textbooks, 
manuals, books, or book-length packs 
of course readings 

b. Number of books read on your own 
(not assigned) for personal enjoyment 


□ 


□ 


□ 


□ 


□ 


or academic enrichment 


□ 


□ 


□ 


□ 


□ 


c. Number of written papers or reports 
of any length 


□ 


□ 


□ 


□ 


□ 



7. Mark the box that best represents the extent to which your examinations during the current school 
year have challenged you to do your best work at this college. 



Extremely 

challenging 



7 6 5 4 3 2 1 

□ □ □ □ □ □ □ 



Extremely easy 



8. Which of the following have you done, are you doing, or do you plan to do while attending this 
college ? 



a. Internship, field experiences, co-op experience, or 
clinical assignment 

b. English as a second language course 

c. Developmental/remedial reading course 



I have not 



I have done 


I plan to 
do 


done nor plan 




to do 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 
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d. Developmental/remedial writing course 




□ 


□ 


□ 


e. Developmental/remedial math course 




□ 


□ 


□ 


f. Study skills course 




□ 


□ 


□ 


g. Honors course 




□ 


□ 


□ 


h. College orientation program or course 

i. Organized learning communities (linked courses/study 


□ 


□ 


□ 


groups led by faculty or counselors) □ 

9. To what extent does this college emphasize each of the following? 


□ 


□ 




Very much 


Quite a bit 


Some 


Very little 


a. Encouraging you to spend significant 
amounts of time studying 


□ 


□ 


□ 


□ 


b. Providing the support you need to help 
you succeed at this college 

c. Encouraging contact among students from 


□ 


□ 


□ 


□ 


different economic, social, and racial or 
ethnic backgrounds 


□ 


□ 


□ 


□ 


d. Helping you cope with your non-academic 
responsibilities (work, family, etc.) 


□ 


□ 


□ 


□ 


e. Providing the support you need to thrive 
socially 


□ 


□ 


□ 


□ 


f. Providing the financial support you need to 
afford your education 


□ 


□ 


□ 


□ 


g. Using computers in academic work 


□ 


□ 


□ 


□ 



10. About how many hours do you spend in a typical 7 -day week doing each of the following? 



a. Preparing for class (studying, reading, 


None 


1-5 


6-10 


11-20 


21-30 


More than 
30 


writing, rehearsing, or other activities 
related to your program) 


□ 


□ 


□ 


□ 


□ 


□ 


b. Working for pay 

c. Participating in college-sponsored 


□ 


□ 


□ 


□ 


□ 


□ 


activities (organizations, campus 
publications, student government, 
intercollegiate or intramural sports, etc.) 
d. Providing care for dependents living 


□ 


□ 


□ 


□ 


□ 


□ 


with your (parents, children, spouse, 
etc.) 


□ 


□ 


□ 


□ 


□ 


□ 


e. Commuting to and from classes 


□ 


□ 


□ 


□ 


□ 


□ 



1 1 . Mark the box that best represents the quality of your relationships with people at this college . 



a. Other Students 


7 


6 


5 


4 


3 


2 


1 




Friendly, 
















Unfriendly, unsupportive. 


supportive, sense 
of belonging 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


sense of alienation 


b. Instructors 


7 


6 


5 


4 


3 


2 


1 




Available, helpful. 
















Unavailable, unhelpful. 


sympathetic 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


unsympathetic 
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c. Administrative Personnel & Offices 
7 6 5 

Helpful, 

considerate n n n 

flexible 



4 3 2 

□ □ □ 



1 

□ 



Unhelpful, inconsiderate, 
rigid 



12. To what extent has YOUR EXPERIENCE AT THIS COLLEGE contributed to your knowledge, 
skills, and personal development in the following areas? 



a. Acquiring a broad general education 

b. Acquiring job or work-related knowledge and skills 

c. Writing clearly and effectively 

d. Speaking clearly and effectively 

e. Thinking critically and analytically 

f. Solving numerical problems 

g. Using computing and information technology 

h. Working effectively with others 

i. Learning effectively on your own 

j. Understanding yourself 

k. Understanding people of other racial or ethnic 
backgrounds 

l. Developing a personal code of values and ethics 

m. Contributing to the welfare of your community 

n. Developing clearer career goals 

o. Gaining information about career opportunities 



Very much 


Quite 
a bit 


Some 


Very little 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



13. This section has three parts. Please answer all three sections, indicating (1) HOW OFTEN you use 
the following services, (2) HOW IMPORTANT the services are to you, and (3) HOW SATISFIED you 
are with the services AT THIS COLLEGE. 



a. Academic advising/planning 
(1) Frequency of Use 



Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


b. Career counseling 








(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


c. Job placement assistance 








(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


d. Peer 


or other tutoring 
(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 



(2) Satisfaction 






(3) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






(3) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






(3) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






(3) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 
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e. Skill labs (writing, math, etc.) 
(1) Frequency of Use 



Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


f. Child care 










(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


g. Financial aid advising 








(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


h. Computer lab 










(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


i. Student organizations 








(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


j. Transfer credit assistance 








(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 


k. Services for people with disabilities 






(1) Frequency of Use 






Often 


Some 

-times 


Rarely/ 

Never 


Don’t 

know/ 

N.A. 


Very 


□ 


□ 


□ 


□ 


□ 



(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 


(2) Satisfaction 






( 3 ) 


Importance 


Some 

-what 


Not at 

all 


N.A. 


Very 




Some 

-what 


Not at 

all 


□ 


□ 


□ 


□ 




□ 


□ 



14. How likely is it that the following issues would cause 
you to withdraw from class or from this college ? 

(Please respond to each item) 



Very 

likely 



Likely 



Some 

-what 

likely 



Not 

likely 



a. Working full-time 

b. Caring for dependents 

c. Academically unprepared 

d. Lack of finances 

e. Transfer to a four-year college or university 



□ 

□ 

□ 

□ 

□ 



□ 

□ 

□ 

□ 

□ 



□ 

□ 

□ 

□ 

□ 



□ 

□ 

□ 

□ 

□ 



15. How supportive are your friends of your attending this college? 

□ Extremely □ Somewhat 

□ Quite a bit □ Not very 
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16. How supportive is your immediate family of your attending this college? 

□ Extremely □ Somewhat 

□ Quite a bit □ Not very 



17. Indicate which of the following are your 
reasons/goals for attending this college 
(Please respond to each item ) 


Primary goal 


Secondary 

goal 


Not a goal 


a. Complete a certificate program 


□ 


□ 


□ 


b. Obtain an associate degree 


□ 


□ 


□ 


c. Transfer to a four-year college or university 


□ 


□ 


□ 


d. Obtain or update job-related skills 


□ 


□ 


□ 


e. Self-improvement/Personal Enjoyment 


□ 


□ 


□ 


f. Change careers 


□ 


□ 


□ 


18. Indicate which of the following are sources you use to pay your tuition 
respond to each item ) 

Major source 


at this college? (Please 

... Nota\ 

Minor source 

source 


a. My own income/savings 


□ 


□ 


□ 


b. Parent or spouse/significant other's income/savings 


□ 


□ 


□ 


c. Employer contributions 


□ 


□ 


□ 


d. Grants & scholarships 


□ 


□ 


□ 


e. Student loans (bank, etc.) 


□ 


□ 


□ 


f. Public assistance 


□ 


□ 


□ 



19. Since high school, which of the following types of schools have you attended other than the one you 
are now attending. (Please mark all that apply) 

□ Proprietary (private) school or training program 

□ Public vocational-technical school 

□ Another community or technical college 

□ Four-year college or university 

□ None 

20. When do you plan to take classes at this college again? 

□ I will accomplish my goal(s) during this term and will not be returning 

□ I have no current plan to return 

□ Within the next 12 months 

□ Uncertain 



21 . 



At this college, in what range is your overall college grade average? 

□ A 

□ A- to B+ 

□ B 

□ B- to C+ 

□ C 

□ C- or lower 

□ Do not have a GPA at this school 

□ Pass/fail classes only 



22. When do you most frequently take classes at this college? (Mark one only) 

□ Day classes (morning or afternoon) 

□ Evening classes 

□ Weekend classes 
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23. How many TOTAL credit hours have you earned at this college, not counting the courses you are 
currently taking this term? 

□ none 

□ 1-14 credits 

□ 15-29 credits 

□ 30-44 credits 

□ 45-60 credits 

□ over 60 credits 



24. At what other types of institutions are you taking classes this term? 

(Please mark all that apply) 

□ None 

□ High school 

□ Vocational/technical school 

□ Another community or technical college 

□ Four-year college/university 

□ Other 

25. How many classes are you presently taking at OTHER institutions? 

□ None 

□ 1 class 

□ 2 classes 

□ 3 classes 

□ 4 classes or more 

26. Would you recommend this college to a friend or family member? 

□ Yes 

□ No 

27. How would you evaluate your entire educational experience at this college? 

□ Excellent 

□ Good 

□ Fair 

□ Poor 

28. Do you have children who live with you? 

□ Yes 

□ No 



29. Mark your age group 



□ 


Under 18 


□ 


18 to 19 


□ 


20 to 21 


□ 


22 to 24 


□ 


25 to 29 


□ 


30 to 39 


□ 


40 to 49 


□ 


50 to 64 


□ 


65+ 


30. Your sex: 




□ 


Male 


□ 


Female 
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31. Are you married? 

□ Yes 

□ No 

32. Is English your native (first) language? 

□ Yes 

□ No 

33. Are you an international student or foreign national? 

□ Yes 

□ No 

34. What is your racial identification? (Mark all that apply ) 

□ American Indian or other Native American 

□ Asian, Asian American, or Pacific Islander 

□ Native Hawaiian 

□ Black or African American 

□ White, Non-Hispanic 

□ Hispanic, Latino, Spanish 

□ Other 

35. What is the highest academic credential you have earned? 

□ None 

□ High school diploma or GED 

□ Vocational/technical certificate 

□ Associate’s degree 

□ Bachelor’s degree 

□ Master’ s/doctoral/professional degree 

36. What is the highest level of education obtained by your: 



a. Not a high school graduate 


Father 

□ 


Mother 

□ 


b. High school diploma or GED 


□ 


□ 


c. Some college, did not complete degree 


□ 


□ 


d. Associate’s degree 


□ 


□ 


e. Bachelor’s degree 


□ 


□ 


f. Master’s degree/ 1st professional 


□ 


□ 


g. Doctorate degree 


□ 


□ 


h. Unknown 


□ 


□ 



Note. From Community College Survey of Student Engagement , by Community College Leadership 
Program (2004), Austin: University of Texas at Austin. Copyright 2004 by Community College Survey of 
Student Engagement. Reprinted with permission. 
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Appendix B 

College Academic Quality Improvement Project Explanation 
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What is AQIP? 

Academic Quality Improvement Project (AQIP) is a new accreditation option that intends 
to revolutionize how the North Central Association (NCA) accredits and reaccredits its 
member colleges. 

It used to be that every 10 years, a team of six or more NCA consultant evaluators would 
read our self-study, make an on-site visit, interview staff, and write a report 
recommending that the college be reaccredited for another 10 years-in the case of a good 
result. AQIP changes all of that. 

In 2000, a group of 14 college staff formed an AQIP study group to take a look at 
whether the new accreditation model would be better or worse for college than the 
traditional process that the college went through in 1995-96 (with 2 Vi years of 
preparation). The group concluded that pursuing AQIP would be a better option for 
college, in part due to its rich history with quality principles and tools. 

AQIP bills itself as a “quality-based” accreditation process, meaning that you can use a 
different set of evaluative criteria than the traditional five criteria used in the past for all 
colleges. With AQIP, a college conducts its own self-assessment, sets its own goals and 
measures itself on how well it is making progress toward achieving the goals. There is 
still a site visit by an outside team, but it occurs once in a seven-year period, supposedly 
without the fanfare that typically accompanies the traditional 10-year visit. 

Per NCA, “AQIP will be nonprescriptive and will not encroach on an institution’s right to 
make its own decisions about its future. . . [it asks] an institution to examine whether it is 
doing those things which will help it achieve its goals, and whether it is doing them as 
effectively as possible.” (NCA AQIP Brochure, page 4) 

NCA developed the AQIP model to offer an alternative for accreditation. Some colleges 
had complained about the traditional NCA model, finding it intrusive and not helpful in 
improvement efforts. Meanwhile, the business world had gravitated toward Malcolm 
Baldrige, state award programs like State Forward, and quality-based evaluation. The 
college’s cross-functional NCA/AQIP team includes staff from many different areas of 
the college. 



Note. Reprinted with permission of subject technical college. 
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Appendix C 

Academic Quality Improvement Project Questions 
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Category One: Helping Students Learn 



Context for Analysis 

1C1 Common Student Learning Objectives: What are the common student learning 
objectives you hold for all of your students (regardless of their status or particular 
program of study) and the pattern of knowledge and skills you expect your students to 
possess upon completion of their general and specialized studies? 

As appropriate, address co-curricular objectives/goals. Category Six, Supporting 
Institutional Operations, asks how you determine, address, and improve your learning 
support systems to contribute to achieving student learning and development objectives. 

1C2 Aligning Student Learning with Mission, Vision, and Philosophy: By what means do 
you ensure that student learning expectations, practices, and development objectives align 
with your mission, vision, and philosophy? 

1C3 Key Instructional Programs: Delivery and Use of Technology: What are your key 
instructional programs? What delivery methods are used within these key programs? To 
what degree is technology used within the formal instructional context? 

1C4 Preparing Students for a Diverse World and Accommodating Various Learning 
Styles: What practices do you use to ensure that the design and delivery of student 
learning options prepare students to live in a diverse world and accommodate a variety of 
student learning styles? 

1C5 Creating a Climate That Celebrates Intellectual Freedom, Reflection, and Respect: 

By what means do you create and maintain a climate that celebrates intellectual freedom, 
inquiry, reflection, respect for intellectual property, and respect for differing and diverse 
opinions? 

Processes 

1P1 Determining Student Learning Objectives and Program Learning Objectives: How 
do you determine your common student learning objectives as well as specific program 
learning objectives? Who is involved in setting these objectives? 

1P2 Designing Responsive Academic Programs: Balancing Education Issues with 
Student Needs: How do you design new programs and courses to facilitate student 
learning? How do you balance educational market issues with student needs in designing 
responsive academic programming? 

1P3 Determining Student Preparation Required for Specific Curricula, Programs, 

Courses, and Learning: How do you determine the preparation required of students for 
the specific curricula, programs, courses, and learning they will pursue? 

1P4 Communicating Expectations of Student Preparation and Learning Objectives: How 
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do you communicate expectations regarding student preparation and student learning 
objectives (for programs, courses, and the awarding of specific degrees or credentials) to 
prospective and current students? 

1P5 Helping Students Select Programs of Study: How do you help students select 
programs of study that match their needs, interests, and abilities? In providing this help, 
how are discrepancies between the necessary and actual preparation of students and their 
learning styles detected and addressed? 

1P6 Determining and Documenting Effective Teaching and Learning: How do you 
determine and document effective teaching and learning? How are these expectations 
communicated across the institution? 

1P7 Effective and Efficient Course Delivery System: How do you build an effective and 
efficient course delivery system? How do delivery decisions balance student and 
institutional needs? 

1P8 Currency and Effectiveness of Curriculum: How do you monitor the currency and 
effectiveness of your curriculum? What process is in place for changing or discontinuing 
programs and courses? 

1P9 Learning Support for Students and Faculty: How do you determine student and 
faculty needs relative to learning support? How are learning support areas involved in the 
student learning and development process? 

1P10 Alignment of Co-curricular Development Goals with Curricular Learning 
Objectives: How are co-curricular development goals aligned with curricular learning 
objectives? 

1P11 Determining Process for Assessment of Student Learning: How do you determine 
the processes for student assessment? 

IP 12 Determining Students’ Preparation for Further Education or Employment: How do 
you discover how well prepared the students who are completing programs, degrees, and 
certificates are for further education or employment? 

1P13 Measures of Student Performance: What measures of student performance do you 
collect and analyze regularly? 

Results 

1R1 Results for Common Student Learning Objectives and Specific Program Learning 
Objectives: What are your results for common student learning objectives as well as 
specific program learning objectives? 



1R2 Evidence of Adequate Student Skills: What is your evidence that students have 
acquired the knowledge and skills base required by the institution and its stakeholders 
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(that is, other educational institutions and employers) for the awarding of specific degrees 
or credentials? Note: Results might address perspectives of other higher education 
institutions, employers, and so on. 

1R3 Results Associated with Helping Students Learn: What are your results for processes 
associated with helping students learn? Note: Results might include processes in 
designing and introducing new courses and programs, using technology and its impact, 
evidence of effective teaching, processes associated with scheduling, and so on. 

Regarding 1R1 through 1R3, how do your results compare with the results of other 
higher education institutions and, if appropriate, organizations outside the education 
community? 

Improvement 

111 Improving Process and Systems: How do you improve your current processes and 
systems for helping students learn and develop? 

112 Targets for Improvement: Determination, Priorities, and Communication: With 
regard to your current results for student learning and development, how do you set 
targets for improvement? What specific improvement priorities are you targeting, and 
how will they be addressed? How do you communicate your current results and 
improvement priorities to students, faculty, staff, administrators, and appropriate 
stakeholders? 



Note. From Principles and Criteria for Improving Academic Quality, by Higher Learning Commission, 
Academic Quality Improvement Program, 2002, Chicago: North Central Association of Colleges and 
Schools. Copyright 2002 by North Central Association of Colleges and Schools. Reprinted with 
permission. 
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Appendix D 

Engagement Tips for Students 
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Tips for Students 

Approaching faculty can be intimidating. But they are people, too! Here are some 
suggested ways to interact with faculty: 

Email 

Contact a professor via email to ask a question or to schedule an appointment. Email 
allows you to "talk" with a professor without worrying about interrupting them, because 
they can answer in their free time. You can give a professor a sense of what you want to 
talk about during your appointment so they have time to prepare information making 
your visit more worthwhile. Email also enables you to take your time with what you want 
to say 

Small talk 

Talk with a professor before class or walk with them back to their office after class 
Invitations 

Invite a professor to attend an event that your club is sponsoring. Hand the professor a 
flyer or an invitation that does the talking for you if you get tongue-tied. 

Office hours 

Stop by during office hours to: 

• Ask how you are doing in class. Ask what you can do better. 

• Ask for more information on a subject in which you are interested. 

• Discuss different paper topic ideas and possible research sources. 

• Let a professor know what you enjoyed about his/her lecture. 

• Find out their recommendations for what classes to take next semester. 

• Ask for their suggestions for an intemship/volunteer position/job in their 
field. 

• Learn their opinions about different graduate programs 

• Volunteer to assist with their research projects 



Note. From Tips, by Arizona State University, (n.d.), retrieved September 24, 2006, from http://www.asu 
.edu/studentaffairs/mu/mpa/tips.htm. Copyright 2006 by Arizona State University. Reprinted with 
permission. 




127 



Appendix E 

Engagement Tips for Faculty 
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Tips for Faculty 

Student-faculty interaction can mean inviting students over to one's house for dinner, 
participating a service project with students, asking students to work on faculty research 
projects, and advising a student organization. 

But interaction can also be defined as informal conversations, including course-related 
topics, self-disclosure, small talk, advice, intellectual ideas, and favor requests. 

Research has demonstrated that students are eager for faculty interaction but that many, 
particularly first-year students, are highly dependent upon faculty encouragement to do 
so. Faculty can encourage students to take the initiative by: 

Availability 

Arrive to class early/stay a few minutes at the end. 

Small talk 

Although the initial function of small talk is to break the ice, it has been demonstrated 
that students feel more confident in the classroom and more valued by the faculty 
member after engaging in small talk . 

Office Hours 

Select student-friendly hours, keep the door open, and suggest reasons why a student 
could stop by, such as to discuss paper topics, how they are doing in the class, and their 
future plans. Faculty can also solicit feedback from their students on the class. 

Email 

Encourage students to email questions, concerns, and thoughts. In one study, students 
who reported being too intimidated to interact in person were able to have frequent 
contact with faculty through email. 

Role as Teacher 

Research shows that when students perceive faculty as making teaching a priority, 
communicating concern that students understand the material, challenging students, 
involving students in the teaching process, and extending the students' understanding 
beyond the class material to include general subject matter, these faculty seemed more 
approachable to students. 

Self disclosure 

When faculty relate to students as 'real' people by communicating interest in students as 
individuals, by talking about themselves, by using humor, or by relating to the student 
perspective, students are more willing to interact with them. Self disclosure can include 
sharing experiences about graduate school, children, and future plans (Daniel-DiGregorio 
1996; Plecha 2002). 

Note. From Tips, by Arizona State University, (n.d.), retrieved September 24, 2006, from http://www.asu 
.edu/studentaffairs/mu/mpa/tips.htm. Copyright 2006 by Arizona State University. Reprinted with 
permission. 
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Appendix F 

Advantages and Disadvantages of Active Learning Techniques 
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Table 1 Advantages and Disadvantages of Active Learning Techniques 



Technique Advantages 



Disadvantages 



Collaborative 

learning 



Problem-based 

learning 

(PBL) 



Service 

Learning 



Involving 
undergraduate 
students in 
research 



Engages students in joint intellectual efforts 

Students learn to work together 

Students are able to maintain some sense of 

individual accountability 

Provides an opportunity for peer evaluations 



Decision about how to form groups 
Getting “buy-in” from students 
Giving up classroom lecture time for 
group work 

Individual members who fail to be 
responsible members of the group 



Sense of commaradie (sic) that is generated among 
student peers 

Challenges students to activate prior knowledge 
Stimulates intrinsic motivation to learn 
A curricular organizer 



Requires careful planning and 
implementation 
More difficult in larger classes 
May need to cut back on content 
coverage tin order to allow PBL 
activities to take place 
Suitable for lower-achieving and 
higher-achieving students 



Provides opportunities for students to apply course 

content to real-world problems 

Allows students to take responsibility for their own 

learning 

Allows for simultaneous service and learning 
Creates a sense of civic responsibility in students 
Allows students to apply major course concepts to 
the community setting 

Can enhance overall scores (examinations, major 
papers, etc.) 

Creates a mentoring relationship between the 
student and the instructor 



Additional effort on the part of the 
student and the instructor 
Identification of sites 
Liability issues related to student 
safety and well being 



Allows a “hands-on” effort related to the learning 
of research methods and statistics 
Travel to professional conferences and 



Requires a high level of motivation 
on the part of both the instructor and 
the student 



enhancement of students’ speaking and networking Additional time commitment on the 



skills 



part of both the student and the 



Encourages students to think about continuing their instructor 
education experience at the graduate level 



Note. From “Creating a Teaching and Learning Environment in criminal Justice Courses That Promotes 
Higher Order Thinking, by B. Sims, (2006), Journal of Criminal Justice Education, 17(2), p. 350. 
Copyright 2006 by the Academy of Criminal Justice Sciences. Reprinted with permission. 
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Appendix G 

Criteria and Research Standards Linkage 
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Utility The utility standards are intended to ensure that an evaluation will serve 

the information needs of the intended users. 

U1 Stakeholder Identification. Persons involved in or affected by the 
evaluation should be identified so that their needs can be addressed. 

U2 Evaluator Credibility. The persons conducting the evaluation should 
be both trustworthy and competent to perform the evaluation so that 
the evaluation findings achieve maximum credibility and acceptance. 

U3 Information Scope and Selection. Information collected should be 
broadly selected to address pertinent questions about the program and 
be responsive to the needs and interests of clients and other specified 
stakeholders. 

U4 Values Identification. The perspectives, procedures, and rationale 
used to interpret the findings should be carefully described so that the 
bases for value judgments are clear. 

U5 Report Clarity. Evaluation reports should clearly describe the program 
being evaluated, including its context, and the purposes, procedures, 
and findings of the evaluation so that essential information is provided 
and easily understood. 

U6 Report Timeliness and Dissemination. Significant interim findings 
and evaluation reports should be disseminated to intended users so 
that they can be used in a timely fashion. 

U7 Evaluation Impact. Evaluations should be planned, conducted, and 
reported in ways that encourage follow-through by stakeholders so 
that the likelihood that the evaluation will be used is increased. 

Feasibility The feasibility standards are intended to ensure that an evaluation will be 

realistic, prudent, diplomatic, and frugal. 

FI Practical Procedures. The evaluation procedures should be practical, 
to keep disruption to a minimum when needed information is 
obtained. 

F2 Political Viability. The evaluation should be planned and conducted 
with anticipation of the different positions of various interest groups 
so that their cooperation may be obtained, and so that possible 
attempts by any of these groups to curtail evaluation operations or to 
bias or misapply the results can be averted or counteracted. 

F3 Cost Effectiveness. The evaluation should be efficient and produce 
information of sufficient value so that the resources expended can be 
justified. 



Propriety The propriety standards are intended to ensure that an evaluation will be 
conducted legally, ethically, and with due regard for the welfare of those 
involved in the evaluation, as well as those affected by its results. 
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PI Sendee Orientation. Evaluations should be designed to assist 

organizations to address and effectively serve the needs of the full 
range of targeted participants. 

P2 Formal Agreements. Obligations of the formal parties to an 

evaluation (what is to be done, how, by whom, when) should be 
agreed to in writing so that these parties are obligated to adhere to all 
conditions of the agreement or formally to renegotiate it. 

P3 Rights of Human Subjects. Evaluations should be designed and 
conducted to respect and protect the rights and welfare of human 
subjects. 

P4 Human Interactions. Evaluators should respect human dignity and 
worth in their interactions with other persons associated with an 
evaluation so that participants are not threatened or harmed. 

P5 Complete and Fair Assessment. The evaluation should be complete 
and fair in its examination and recording of strengths and weaknesses 
of the program being evaluated so that strengths can be built upon 
and problem areas addressed. 

P6 Disclosure of Findings. The formal parties to an evaluation should 
ensure that the full set of evaluation findings along with pertinent 
limitations are made accessible to the persons affected by the 
evaluation, and any others with expressed legal rights to receive the 
results. 

P7 Conflict of Interest. Any conflict of interest should be dealt with 
openly and honestly so that it does not compromise the evaluation 
processes and results. 

P8 Fiscal Responsibility. The evaluator’s allocation and expenditure of 
resources should reflect sound accountability procedures and 
otherwise be prudent and ethically responsible so that expenditures 
are accounted for and appropriate. 

Accuracy The accuracy standards are intended to ensure that an evaluation will 

reveal and convey technically adequate information about the features that 

determine worth of merit of the program being evaluated. 

A1 Program Documentation. The program being evaluated should be 
described and documented clearly and accurately so that the program 
is clearly identified. 

A2 Context Analysis. The context in which the program exists should be 
examined in enough detail so that its likely influences on the program 
can be identified. 

A3 Described Purposes and Procedures. The purposes and procedures 
of the evaluation should be monitored and described in enough detail 
so that they can be identified and assessed. 

A4 Defensible Information Sources. The sources of information used in a 
program evaluation should be described in enough detail so that the 
adequacy of the information can be assessed. 

A5 Valid Information. The information gathering procedures should be 
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chosen or developed and then implemented so that they will assure 
that the interpretation arrived at is valid for the intended use. 

A6 Reliable Information. The information gathering procedures should 
be chosen or developed and then implemented so that they will 
assure that the information obtained is sufficiently reliable for the 
intended use. 

A7 Systematic Information. The information collected, processed, and 
reported in an evaluation should be systematically reviewed and any 
errors found should be corrected. 

A8 Analysis of Quantitative Information. Quantitative information in an 
evaluation should be appropriately and systematically analyzed so 
that the evaluation questions are effectively answered. 

A9 Analysis of Qualitative Information. Qualitative information in an 
evaluation should be appropriately and systematically analyzed so 
that the evaluation questions are effectively answered. 

A10 Justified Conclusions. The conclusions reached in an evaluation 
should be explicitly justified, so that stakeholders can assess them. 

All Impartial Reporting. Reporting procedures should guard against 
distortion caused by personal feelings and biases of any party to the 
evaluation so that evaluation reports fairly reflect the evaluation 
findings. 

A12 Metaevaluation. The evaluation itself should be formatively and 

summatively evaluated against these and other pertinent standards so 
that its conduct is appropriately guided and, on completion, 
stakeholders can closely examine its strengths and weaknesses. 



Note. From The Program Evaluation Standards: How to Assess Evaluations of Educational Programs (2nd 
ed.), by the Joint Committee on Standards for Educational Evaluation Programs, 1994, Thousand Oaks, 

CA: Sage. Copyright 1994 by Sage Publications. Reprinted with permission. 
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Student Survey 

1. During this school year, how regularly have you done the following in your criminal justice 
classes? 



a. Contributed to class discussions and/or asked 
questions in class 

b. Made a presentation to your class 

c. Drafted a paper or assignment at least twice 
before turning it in 

d. Integrated concepts or facts from various 
sources in a paper or project for class 

e. Failed to complete readings or assignments 
prior to class 

f. Prepared projects with other students during 
class 

g. Prepared assignments with other students 
outside of class 

h. Explained concepts or ideas to other students 
in preparation for class 

i. As an assignment/project for a class, 
participated in a community-based project 

j. Used the internet or instant messaging to 
prepare an assignment 

k. Communicated with an instructor via e-mail 

l. Talked to an instructor about your grades or 
assignments 

m. Held a discussion with an instructor or advisor 
regarding your career plans 

n. Continued classroom discussions or discussed 
ideas from your readings with instructors after 
the class period ended 

o. Instructors provide written or oral feedback 
regarding your performance promptly 

p. Exceeded your own expectations, or worked 
beyond what you believed were your capabilities 
to meet an instructor's standards or expectations 

q. Interacted with instructors on activities other 
than coursework 

r. Found yourself engaging other students, family 
or co-workers in discussions centered on your 
readings or classes 

s. Seriously discussed ideas with students of a 
different race or ethnicity 

t. Seriously discussed ideas with students holding 
different religious beliefs, political opinions, or 
personal values than those held by you 

u. Failed to attend class without an excuse 



Very 




Some- 




Often 


Often 


times 


Never 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



2. During this school year, what emphasis has your criminal justice coursework placed on the 
following? 
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Very 





much 


Quite a bit 


Some 


Very little 


a. Memorization: Repeating facts, ideas, or 
methods from your courses and readings in 
practically the same form in which they were 
presented 

b. Analysis: breaking down an idea, experience 
or theory to its basic elements 

c. Perspective: finding new ways to organize and 
integrate ideas, information, or experiences 

d. Evaluation: assessing the value or soundness 
of information, arguments, or methods 

e. Adaptation: finding new applications of 
theories or concepts to practical problems or in 
new situations 

f. Proficiency: performing a new skill using 
information you have read or heard 


□ 


□ 


□ 


□ 










□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 










□ 


□ 


□ 


□ 










□ 


□ 


□ 


□ 


3. What amount of reading and writing have you done during this school year? 




None 


1 to 4 


5 to 10 


1 1 to 20 


More than 20 


a. Assigned texts, books, manuals, or ^ 

book-length course readings _ 


□ 


□ 


□ 


□ 


b. Books read on your own for personal ^ 

enjoyment or academic enrichment 


□ 


□ 


□ 


□ 


c. Assigned written papers or reports □ 


□ 


□ 


□ 


□ 


4. How challenging have your criminal justice examinations been during the current school year? 


7 6 5 4 


3 2 


1 






Extremely difficult □ □ □ □ 


□ □ 


□ Extremely undemanding 



5. How much emphasis does the criminal justice department place on each of the following? 

a. Spending substantial amounts of time 
studying 

b. Supporting your success in the program 

c. Encouraging students to interact with those of 
diverse economic, social, racial, or ethnic 
backgrounds 

d. Assistance with your non-academic 
responsibilities to your family, employer, etc. 

e. Support for social interaction and growth 

f. Making you aware of financial support 
opportunities to help finance your education 

g. Computer uses in your schoolwork 



Very much 


Quite a bit 


Some 


Very little 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



6. During a full seven-day week, how much time, in hours, do you normally spend on each of the 
following activities? 



None 



1-5 



6-10 11-20 21-30 More than 
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30 



a. Preparing for class 


□ 


□ 


□ 


□ 


□ 


□ 


b. Employment for pay 


□ 


□ 


□ 


□ 


□ 


□ 


c. Participating in college-sponsored 
activities outside the classroom 


□ 


□ 


□ 


□ 


□ 


□ 


d. Providing care for family members in 
your home 


□ 


□ 


□ 


□ 


□ 


□ 


e. Travel to and from classes 


□ 


□ 


□ 


□ 


□ 


□ 



7. How would you rate your relationships with others in the criminal justice department? 

a. Other Students 





7 


6 


5 


4 


3 


2 


1 




Friendly, 
















Unfriendly, unsupportive. 


supportive, sense of 
camaraderie 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


sense of hostility 


b. Instructors 


7 


6 


5 


4 


3 


2 


1 




Available, 
















Unavailable, unsupportive, 


supportive, sensitive 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


insensitive 


c. Office and Administrative Personnel 














7 


6 


5 


4 


3 


2 


1 




Helpful, 

understanding, 

flexible 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unhelpful, uncaring, rigid 



8. Have your skills and personal development in the following areas been improved by the criminal 
justice department? 



a. A broad general education 

b. Job or work-related knowledge and skills 

c. Clear and effective writing skills 

d. Clear and effective speaking skills 
e Critical and analytical thinking 

f. Numerical problem-solving skills 

g. Information technology and computer skills 

h. Ability to work effectively with others 

i. Skills to learn effectively on your own 

j. Understanding yourself 

k. Understanding people of diverse racial/ethnic 
backgrounds 

l. Personal code of values and ethics development 

m. Contributing to community welfare 

n. Refining career goals 

o. Career opportunity information 

9. How often are you involved with student 
organizations? 



Very much 


Quite a 
bit 


Some 


Very li 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Often Some- Rarely/Never Don’t know/ 
times N.A. 

□ □ □ □ 



10. How satisfied are you with the student 
organizations? 



Very 



Some- 

what 



Not at all 



□ □ □ 



N.A. 

□ 
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11. How important are student organizations to you? Very Somewhat Not at all 

□ □ □ 

12. How would you characterize your friends’ support of your involvement in criminal justice 
classes? 

□ Extremely supportive □ Somewhat supportive 

□ Quite supportive □ Not very supportive 

13. How would you characterize your immediate family’s support of your involvement in criminal 
justice classes? 

□ Extremely supportive □ Somewhat supportive 

□ Quite supportive □ Not very supportive 



14. Why are you taking criminal justice classes? (Please 
respond to each item) 


Primary goal 


Secondary 

goal 


Not a goal 


a. Complete a certificate program 


□ 


□ 


□ 


b. Obtain an associate degree 


□ 


□ 


□ 


c. Transfer to a four-year college or university 


□ 


□ 


□ 


d. Obtain or update job-related skills 


□ 


□ 


□ 


e. Self-improvement/personal enjoyment 


□ 


□ 


□ 


f. Change careers 


□ 


□ 


□ 


15. How are you financing your education? (Please respond . , . 

, . ^ Maior source 

to each item ) 


Minor source 


Not a 
source 


a. My own income/savings 


□ 


□ 


□ 


b. Parent or spouse/significant other's income/savings 


□ 


□ 


□ 


c. Employer contributions 


□ 


□ 


□ 


d. Grants & scholarships 


□ 


□ 


□ 


e. Student loans 


□ 


□ 


□ 


f. Public assistance 


□ 


□ 


□ 



16. What is your overall grade average? 

□ A 

□ A- to B+ 

□ B 

□ B- to C+ 

□ C 

□ C- or lower 

□ Do not have a GPA at this school 

□ Pass/fail classes only 



17. My educational experience in the criminal justice department has been: 

□ Excellent 

□ Good 

□ Fair 

□ Poor 



18. Age 

□ Under 18 

□ 18 to 19 
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□ 20 to 21 

□ 22 to 24 

□ 25 to 29 

□ 30 to 39 

□ 40 to 49 

□ 50 to 64 

□ 65+ 

19. Sex 

□ Male 

□ Female 

20. Marital status 

□ Married 

□ Single (never married) 

□ Divorced 

□ Widowed 

21. Racial grouping (Mark all that apply) 

□ American Indian or other Native American 

□ Asian, Asian American or Pacific Islander 

□ Native Hawaiian 

□ Black or African American 

□ White, Non-Hispanic 

□ Hispanic, Latino, Spanish 

□ Other 

22. Parents’ education 

Father Mother 



a. Not a high school graduate □ 

b. High school diploma or GED □ 

c. Some college, did not complete degree □ 

d. Associate’s degree □ 

e. Bachelor’s degree □ 

f. Master’s degree/first professional □ 

g. Doctorate degree □ 

h. Unknown □ 



23. Are you the first member of your family to attend college? 

□ Yes 

□ No 



□□□□□□□□ 
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Survey Participation Invitation 
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Month, Day, 2007 



Dear Survey Participant, 

My name is Marty Totzke, and I am a doctoral student attending Nova 
Southeastern University. As a requirement of my doctoral program, I am conducting an 
applied dissertation study and am writing to invite you to participate by completing a 
survey pertaining to the topic of my study. 

My applied dissertation study focuses on student engagement at a technical 
college criminal justice program. Student engagement is the involvement of students in 
the learning process. Engagement includes preparation for class, out-of-class discussions, 
and any other social or academic activity that contributes to learning of academic 
material. Once obtained, the feedback will be interpreted and shared with individuals 
within the organization for the purpose of preparing a model of best practices to be used 
by instructors. 

You have been selected for this study because you are an active student in the 
criminal justice program. By participating in the study, consent is given that the data may 
be used for research purposes. Confidentiality will be protected. 

Thank you for your consideration of my invitation. I will contact you within the 
next week to answer any questions that you may have and to learn of your decision. 



Marty Totzke, Doctoral Student 
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Appendix J 

Formative and Summative Committee Members 
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Formative Committee Members 



Student Counselor 

Qualifications: B.A. Psychology, University of Wisconsin-Stevens Point; M.S. 
Educational Counseling, University of Wisconsin-Oshkosh, Ed. D., Higher Education 
Leadership, Nova Southeastern University. 

Reason for selection: This member is the counselor for the criminal justice program and 
receives critical feedback from students regarding the quality of program instruction. 

Formative committee selection process: Discussion explaining the purpose of the 
dissertation and the role of the formative committee. 

Criminal Justice Adjunct Faculty Member 

Qualifications: A.A.S. Police Science, Fox Valley Technical College; B.A., Leadership in 
Public Service, Silver Lake College; M.S., Organizational Behavior, Silver Lake College. 

Reason for selection: This member’s elective credit focused on adult learning theory. The 
member proactively incorporated student engagement activities in the classroom. 

Formative committee selection process: Discussion explaining the purpose of the 
dissertation and the role of the formative committee. 

Criminal Justice Instructor 

Qualifications: B.S., Criminal Justice, University of Wisconsin-Oshkosh; M.Ed., 

Northern Arizona University. 

Reason for selection process: This member was chosen due to his Master of Education 
degree. The member holds an Arizona Community College instructor certification in the 
field of Administration of Justice. The member served as adjunct faculty at three 
institutions. 

Formative committee selection process: Discussion explaining the purpose of the 
dissertation and the role of the formative committee. 
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Members of the Summative Committee 

Vice President of Instruction and Academic Dean of a 4-year Midwestern college 

Qualifications: B.S., Business Education, University of Wisconsin- Whitewater; M.S. 
Distributive Education, University of Wisconsin; Ed. D., Vocational Education, Nova 
Southeastern University. 

Reason for selection to the Summative Committee: This member is also a retired Dean of 
Business and Marketing from a State Technical College. Having received a Doctorate 
from NOVA, this member is familiar with the committee process. 

Summative Committee Selection Process: Discussion explaining the dissertation purpose 
and the role of a summative committee member. 

Associate Dean of Criminal Justice of a State Technical College. 

Qualifications: B.S., Criminal Justice, University of Wisconsin-Oshkosh; M.S., 
Educational Leadership, Marian College. 

Reason for selection to the Summative Committee: As a Dean of a Criminal Justice 
program in a similarly situated technical college, this member contributes administrative 
implementation insight from the perspective of a technical college dean. 

Summative Committee Selection Process: Discussion explaining the dissertation purpose 
and the role of a summative committee member. 

Social Science Instructor and Retired Criminal Justice Manager of a State Technical 
College. 

Qualifications: B.S., Business Education, University of Nebraska; M.S.W., University of 
Nebraska. 

Reason for selection to the Summative Committee: This member possesses an enormous 
capacity for detail and critical evaluation. 

Summative Committee Selection Process: Discussion explaining the dissertation purpose 
and the role of a summative committee member 
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Faculty Survey 
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Faculty Survey 



1. During this school year, how regularly have you had students do the following in your criminal 
justice classes? 



a. Contribute to class discussions and/or ask 
questions in class 

b. Make a presentation in your class 

c. Prepare more than one draft of a paper or 
assignment before turning in the final work 

d. Integrate concepts or facts from various 
sources in a paper or project for class 

e. Fail to complete readings or assignments prior 
to class 

f. Prepare projects with other students during 
class 

g. Prepare assignments with other students 
outside of class 

h. Explain concepts or ideas to other students in 
preparation for class 

i. As an assignment/project for a class, participate 
in a community-based project 

j. Use the internet or instant messaging to prepare 
an assignment 

k. Communicate with you via e-mail 

l. Talk to you about their grades or assignments 

m. Hold a discussion with you regarding their 
career plans 

n. Continue classroom discussions or discuss 
ideas from the readings with you after the class 
period ended 

o. Request prompt written or oral feedback 
regarding their performance 

p. Exceed your own expectations, or work 
beyond what you believed were their capabilities 
to meet your standards or expectations 

q. Interact with you on activities other than 
coursework 

r. Find them engaging other students, family or 
co-workers in discussions centered on their 
readings or classes 

s. Seriously discuss ideas with students of a 
different race or ethnicity 

t. Seriously discuss ideas with students holding 
different religious beliefs, political opinions, or 
personal values than those held by themselves 

u. Fail to attend class without an excuse 



Very 




Some- 




Often 


Often 


times 


Never 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



2. During this school year, what emphasis have you placed on the following? 

Very 

much Quite a bit Some Very little 

a. Memorization: Repeating facts, ideas, or □ □ □ □ 
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methods from courses and readings in practically 
the same form in which they were presented 

b. Analysis: breaking down an idea, experience 
or theory to its basic elements 

c. Perspective: finding new ways to organize and 
integrate ideas, information, or experiences 

d. Evaluation: assessing the value or soundness 
of information, arguments, or methods 

e. Adaptation: finding new applications of 
theories or concepts to practical problems or in 
new situations 

f. Proficiency: performing a new skill using 
information students have read or heard 



□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



3. What amount of reading and writing do you believe students have done during this school year? 

a. Assigned texts, books, manuals, or 
book-length course readings 

b. Books read on their own for personal 
enjoyment or academic enrichment 

c. Assigned written papers or reports 



None 


1 to 4 


5 to 10 


1 1 to 20 


More than 20 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



4. How challenging have you made your examinations during the current school year? 

7 6 5 4 3 2 1 

Extremely difficult □ □ □ □ □ □ □ Extremely undemanding 



5. How much emphasis do you place on each of the following? 



a. Spending substantial amounts of time 
studying 

b. Supporting students’ success in the program 

c. Encouraging students to interact with those of 
diverse economic, social, racial, or ethnic 
backgrounds 

d. Assistance with students’ non-academic 
responsibilities to your family, employer, etc. 

e. Support for social interaction and growth 

f. Making students aware of financial support 
opportunities to help finance their education 

g. Computer uses in students’ school work 



Very much 


Quite a bit 


Some 


Very little 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



6. During a full seven-day week, how much time, in hours, do you believe your students normally 
spend on each of the following activities? 





None 


1-5 


6-10 


11-20 


21-30 


More than 
30 


a. Preparing for class 


□ 


□ 


□ 


□ 


□ 


□ 


b. Employment for pay 


□ 


□ 


□ 


□ 


□ 


□ 


c. Participating in college-sponsored 
activities outside the classroom 


□ 


□ 


□ 


□ 


□ 


□ 


d. Providing care for family members in 
their home 


□ 


□ 


□ 


□ 


□ 


□ 


e. Travel to and from classes 


□ 


□ 


□ 


□ 


□ 


□ 
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7. How would you rate your students’ relationships with others in the criminal justice department? 

a. Other Students 





7 


6 


5 


4 


3 


2 


1 




Friendly, 

supportive, sense of 
camaraderie 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unfriendly, unsupportive, 
sense of hostility 


b. Instructors 


7 


6 


5 


4 


3 


2 


1 




Available, 

supportive, sensitive 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unavailable, unsupportive, 
insensitive 


c. Office and Administrative Personnel 














7 


6 


5 


4 


3 


2 


1 




Helpful, 

understanding. 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


Unhelpful, uncaring, 
rigid 



flexible 



8. Have your students’ skills, and personal development in the following areas been improved by 
the criminal justice department? 



a. A broad general education 

b. Job or work-related knowledge and skills 

c. Clear and effective writing skills 

d. Clear and effective speaking skills 
e Critical and analytical thinking 

f. Numerical problem-solving skills 

g. Information technology and computer skills 

h. Ability to work effectively with others 

i. Skills to learn effectively on their own 

j. Understanding themselves 

k. Understanding people of diverse racial/ethnic 
backgrounds 

l. Personal code of values and ethics development 

m. Contributing to community welfare 

n. Refining career goals 

o. Career opportunity information 

9. How often are you involved with student 
organizations? 



10. How satisfied are you with the support 
student organizations provide your 
students? 



Very much 


Quite a 
bit 


Some 


Very little 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 


□ 



Often Sometimes Rarely/Never Don’t know/ 

N.A. 

□ □ □ □ 



Very Somewhat Not at all N.A. 

□ □ □ □ 



11. How important are student organizations to your Very Somewhat Not at all 

students? □ □ □ 

12. How would you characterize the support your students receive from their friends for their 
involvement in criminal justice classes? 

□ Extremely supportive □ Somewhat supportive 

□ Quite supportive □ Not very supportive 
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13. How would you characterize the support your students receive from their families for their 
involvement in criminal justice classes? 

□ Extremely supportive □ Somewhat supportive 

□ Quite supportive □ Not very supportive 



14. Regarding each of the following, how would you 



characterize your students’ goals for taking criminal 
justice classes? 


Primary goal 


Secondary 

goal 


Not a goal 


(Please respond to each item ) 








a. Complete a certificate program 


□ 


□ 


□ 


b. Obtain an associate degree 


□ 


□ 


□ 


c. Transfer to a four-year college or university 


□ 


□ 


□ 


d. Obtain or update job-related skills 


□ 


□ 


□ 


e. Self-improvement/personal enjoyment 


□ 


□ 


□ 


f. Change careers 


□ 


□ 


□ 


15. How do you believe your students are financing their 
education? (Please respond to each item ) 


Major source 


Minor source 


Not a 
source 


a. Personal income/savings 


□ 


□ 


□ 


b. Parent or spouse/significant other's income/savings 


□ 


□ 


□ 


c. Employer contributions 


□ 


□ 


□ 


d. Grants & scholarships 


□ 


□ 


□ 


e. Student loans 


□ 


□ 


□ 


f. Public assistance 


□ 


□ 


□ 



16. How do you believe your students would rate their educational experience in the criminal justice 
department? 

□ Excellent 

□ Good 

□ Fair 

□ Poor 
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Appendix L 
Dissemination Protocol 
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Dissemination Protocol 

1. The researcher will present the validated evaluation study results and 
recommendations to the dean of the criminal justice division and the executive 
vice president/chief academic officer at an individually scheduled meeting. 

2. The dean of criminal justice and the executive vice president/chief academic 
officer will review the results and recommendations from the study. 
Recommendations will be reviewed, analyzed, and considered based on the 
mission of the college, mission of the criminal justice division, and budget and 
personnel constraints. 

3. The recommendations accepted by the dean of criminal justice and the executive 
vice president/chief academic officer will be prioritized and a contact person 
assigned to be responsible for assisting in the development and implementation of 
the assigned recommendations. The dean of the criminal justice will provide 
leadership in the implementation of the student engagement recommendations. 

4. The dean of criminal justice will meet with the faculty to discuss the 
recommendations. The discussion will be ongoing until recommendations have 
been assigned to specific faculty members. 

5. The criminal justice dean will work with these assigned personnel to complete an 
implementation plan for their specific duties as assigned. This plan will include a 
list of roles, responsibilities, resources, budget, and timeline. 
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Appendix M 
Evaluation Instrument 
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Evaluation Instrument 



Section 1 

Expected Outcomes (Criteria) for the Student Engagement Process in the 
Criminal Justice Program Currently Utilized by the Criminal Justice Division. 
(What the student engagement process at the college is expected to be doing 

right now) 



1. Involves students in cocurricular activities. 



Do the current engagement practices encourage student participation in activities 
outside of the classroom? These activities may relate to criminal justice studies or any 
other activity that will encourage student involvement and discussion of academic or 
social issues. 



2. Facilitates student interaction. 



Do current practices encourage students to interact with their instructors? Are 
instructors available during non-classroom hours to meet with students? Do the 
practices facilitate peer interaction? 



3. Participate in activities that promote engagement. 



Do courses include activities that include problem solving, or require critical thinking 
by students individually or in a team environment? Courses encourage student 
participation in college co-curricular activities as well as community based 
involvement. 



4. Creates an environment where students and faculty interact. 



Do the current engagement practices create an environment where students feel faculty 
members are approachable? 
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5. Increases critical thinking skills. 



Do the engagement practices include reality-based scenarios that require students to 
evaluate an issue and arrive at a reasonable conclusion? 



6. Creates an atmosphere for ethnically diverse information exchanges. 



Do current practices create opportunities for students to interact with students and 
citizens of diverse backgrounds? 



7. Increases student interaction in class activities. 



Course projects and activities are designed to facilitate in-class student and faculty 
interaction. 
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Section II 

Expected Criteria for the Student Engagement Process in the Criminal Justice 
Program Currently Utilized by the College. 



This section may include all criteria of the criminal justice division student 
engagement practices and may have additional criteria not identified in the 
division student engagement practices. 



Criteria 



Components/Activities Tied to Criteria 



1. Engagement is effected by 
socioeconomic factors. 



Student employment and family obligations were 
surveyed. 

• The effect of generational factors on student 
engagement. 



2. Importance of engagement 
practices is noted by faculty. 



• Faculty identify course assignments that 
encourage student engagement. 

• Faculty facilitate opportunities to interact with 
students in a variety of settings. 



3. Student engagement activities 
are well crafted. 



• Identified engagement activities and practices 
area designed to fulfill an .identified objective. 

• Engagement activities are constructed to use 
adult learning principles. 

• Engagement activities stimulate the affective, 
cognitive and behavioral elements of learning. 



4. Individual meetings occur 
between faculty and students. 



• Students and faculty report meeting individually. 
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Criteria 


Components/Activities Tied to Criteria 


5. Engagement practices 
include past experiences of 
students. 


• Students are encouraged to include life 
experiences in learning activities. 


6. Included in courses are 
situational learning activities 


• Problem solving and decision making is a 
component of assigned activities. 


7. Program curriculum is 
structured to form learning 
communities. 


• Students are required to attended courses as 
cohorts. 


8. Courses contain experiential 
learning components. 


• Assigned projects contain a practical 
application component. 
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Section III 

Evaluation and Gap Analysis 

This section will evaluate the validated criteria of the current student engagement 
practices use by the criminal justice department. The criteria will be evaluated processes 
and practices identified in the literature review. The researcher has identified criteria of 
student engagement practices at the criminal justice division of the college. These 
practices will be compared in order to create a gap analysis. The gap analysis will be 
completed and viewed by an expert evaluation panel. 

Evaluators, please mark the appropriate number demonstrating the ability of the college 
peer advising services meeting the criteria in comparison to best practices as listed in 
Section II of this document. 



Criteria to be compared ^ 

oSo 

S3 

The current student engagement process 2 « 

in the criminal justice department: ^ o3o 

io -< 

1. Facilitates student interaction. 5 4 3 2 1 



Comments for 1 . 



2. Participate in activities that 5 4 3 2 1 

promote engagement. 




Comments for 2. 





2=Disagree 




Criteria to be compared 



The current student engagement process in the 
criminal justice department: 



oSo 

a 

2 £ 

C/D 

M “ 
IT) < 



6. Increases student interaction in class 
activities. 



Comments for 6. 



7. Engagement is effected by 
socioeconomic factors. 



Comments for 7 . 



8. Importance of engagement practices is 5 4 3 

noted by faculty. 



Comments for 8. 



9. Student engagement activities are well 
crafted. 






2=Disagree 




Criteria to be compared 








© 

2 




© 

© 






The current student engagement process in the 


OX 

S 

© 

•- 


© 

© 


© 

© 

OX 


‘© 

© 

-o 

© 


2 

■4-J 


© 

0X1 

CS 

<Z3 


ox 

© 

© 

© 


0> 
0 ) 

u 

SJD 


criminal justice department: 


m 

ii 


© 

OX 


< 

ll 


P 

ii 


3 

© 


5 

II 


in 


C3 

yi 




irj 


< 


-t 




Z 


d 


rH 


s 


10. Individual meetings occur between 




5 


4 




3 


2 




1 



faculty and students. 
Comments for 10. 



11. Engagement practices include past 5 4 3 2 1 

experiences of students. 

Comments for 11. 



12. Collaboration is included in projects that 

involve problem solving. 5 4 3 2 1 

Comments for 12. 



13. Included in courses are situational 

learning activities. 5 4 3 2 1 



Comments for 13. 
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Criteria to be compared 

The current student engagement process in the 
criminal justice department: 





14. Program curriculum is structured to form 5 4 3 2 1 

learning communities. 



Comments for 14. 



15. Courses contain experiential learning 5 4 3 2 1 

components. 



Comments for 15. 



2=Disagree 
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Appendix N 
Student Survey Results 
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Student Survey Results 

1. During this school year, how regularly have you done the following in your criminal justice 
classes? 

Very Some- 





Often 


Often 


times 


Never 


a. Contributed to class discussions and/or asked questions in class 


30% 


40% 


28% 


3% 


b. Made a presentation to your class 


18% 


43% 


38% 


3% 


c. Drafted a paper or assignment at least twice before tinning it in 

d. Integrated concepts or facts from various sources in a paper or 


23% 


40% 


28% 


8% 


project for class 


18% 


43% 


35% 


5% 


e. Failed to complete readings or assignments prior to class 


3% 


15% 


35% 


48% 


f. Prepared projects with other students during class 


8% 


46% 


46% 


0% 


g. Prepared assignments with other students outside of class 

h. Explained concepts or ideas to other students in preparation for 


5% 


40% 


48% 


8% 


class 

i. As an assignment/project for a class, participated in a 


13% 


25% 


58% 


5% 


community-based project 


5% 


23% 


60% 


13% 


j. Used the internet or instant messaging to prepare an assignment 


23% 


40% 


25% 


13% 


k. Communicated with an instructor via e-mail 


28% 


28% 


38% 


8% 


l. Talked to an instructor about your grades or assignments 

m. Held a discussion with an instructor or advisor regarding your 


18% 


33% 


45% 


5% 


career plans 

n. Continued classroom discussions or discussed ideas from your 


13% 


15% 


40% 


33% 


readings with instructors after the class period ended 
o. Instructors provide written or oral feedback regarding your 


3% 


23% 


58% 


18% 


performance promptly 

p. Exceeded your own expectations, or worked beyond what you 
believed were your capabilities to meet an instructor's standards 


18% 


43% 


40% 


0% 


or expectations 


8% 


53% 


38% 


3% 


q. Interacted with instructors on activities other than coursework 

r. Found yourself engaging other students, family or co-workers 


8% 


30% 


53% 


10% 


in discussions centered on your readings or classes 
s. Seriously discussed ideas with students of a different race or 


18% 


40% 


35% 


8% 


ethnicity 

t. Seriously discussed ideas with students holding different 
religious beliefs, political opinions, or personal values than those 


8% 


20% 


55% 


18% 


held by you 


8% 


25% 


48% 


20% 


u. Failed to attend class without an excuse 


0% 


3% 


18% 


80% 
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2. During this school year, what emphasis has your criminal justice coursework placed on the 
following? 





Very much 


Quite a bit 


Some 


Very little 


a. Memorization: Repeating facts, ideas, or 
methods from your courses and readings in 
practically the same form in which they were 
presented 

b. Analysis: breaking down an idea. 


21% 




51% 


28% 


0% 


experience or theory to its basic elements 
c. Perspective: finding new ways to organize 
and integrate ideas, information, or 


18% 




45% 


34% 


3% 


experiences 


33% 




21% 


36% 


10% 


d. Evaluation: assessing the value or 
soundness of information, arguments, or 
methods 


21% 




33% 


46% 


0% 


e. Adaptation: finding new applications of 
theories or concepts to practical problems or 
in new situations 


23% 




28% 


44% 


5% 


f. Proficiency: performing a new skill using 
information you have read or heard 


46% 




38% 


15% 


0% 


3. What amount of reading and writing have you done during this school year? 














More than 




None 


1 to 4 


5 to 10 


11 to 20 20 


a. Assigned texts, books, manuals, or 
book-length course readings 

b. Books read on your own for personal 


3% 


51% 


28% 


15% 


3% 


enjoyment or academic enrichment 


28% 


55% 


10% 


3% 


5% 


c. Assigned written papers or reports 


0% 


30% 


38% 


23% 


10% 


4. How challenging have your criminal justice examinations been during the current 


school year? 


Extremely 










Extremely 


Difficult 










Undemanding 


7 6 


5 


4 


3 


2 


1 


3% 5% 


50% 


30% 8% 


3% 


3% 
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5. How much emphasis does the criminal justice department place on each of the following? 



a. Spending substantial amounts of time 


Very much 


Quite a bit 


Some 


Very little 


studying 

b. Supporting your success in the 


0% 


45% 


53% 


3% 


program 

c. Encouraging students to interact with 
those of diverse economic, social, racial, 


40% 


48% 


13% 


0% 


or ethnic backgrounds 

d. Assistance with your non-academic 

responsibilities to your family. 


30% 


23% 


38% 


10% 


employer, etc. 

e. Support for social interaction and 


3% 


40% 


33% 


25% 


growth 

f. Making you aware of financial 
support opportunities to help finance 


20% 


30% 


43% 


8% 


your education 


30% 


28% 


25% 


18% 


g. Computer uses in your school work 


48% 


35% 


13% 


5% 



6. During a full seven-day week, how much time, in hours, do you normally spend on each of the 
following activities? 



a. Preparing for class 

b. Employment for pay 

c. Participating in college-sponsored 
activities outside the classroom 

d. Providing care for family 
members in your home 

e. Travel to and from classes 



None 


1 to 5 


6 to 10 


5% 


70% 


18% 


8% 


5% 


5% 


60% 


35% 


3% 


25% 


40% 


10% 


3% 


65% 


20% 



11 to 20 


21 to 30 


More than 
30 


8% 


0% 


0% 


20% 


8% 


55% 


0% 


3% 


0% 


5% 


3% 


18% 


8% 


5% 


0% 



7. How would you rate your relationships with others in the criminal justice department? 



a. Other Students 



Friendly, supportive, sense 

of camaraderie 7 6 5 



4 3 2 



25% 



43% 20% 10% 3% 0% 



Unfriendly, 
unsupportive, sense of 
hostility 
1 

0% 



b. Instructors 



Available, supportive, 

sensitive 7 6 5 



Unavailable, 
unsupportive, insensitive 
4 3 2 1 



30% 



43% 25% 3% 0%0% 



0% 



Helpful, understanding, 
flexible 
7 



6 5 4 3 2 



Unhelpful, uncaring, 
rigid 
1 



25% 



38% 20% 18% 0%0% 



0% 



c. Office and 

Administrative 

Personnel 
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8. Have your, skills, and personal development in the following areas been improved by the criminal 
justice department? 



a. A broad general education 


Very much 

49% 


Quite a 

36% 


bitSome 

13% 


Very Little 

3% 


b. Job or work-related knowledge and 
skills 


69% 


26% 


5% 


0% 


c. Clear and effective writing skills 


33% 


44% 


23% 


0% 


d. Clear and effective speaking skills 


33% 


49% 


18% 


0% 


e Critical and analytical thinking 


49% 


36% 


15% 


0% 


f. Numerical problem-solving skills 


28% 


21% 


33% 


18% 


g. Information technology and 
computer skills 


18% 


36% 


26% 


21% 


h. Ability to work effectively with 
others 


41% 


38% 


21% 


0% 


i. Skills to learn effectively on your 
own 


31% 


41% 


26% 


3% 


j. Understanding yourself 


21% 


46% 


26% 


8% 


k. Understanding people of diverse 
racial/ethnic backgrounds 


10% 


51% 


33% 


5% 


1. Personal code of values and ethics 
development 


36% 


41% 


23% 


0% 


m. Contributing to community welfare 


21% 


46% 


31% 


3% 


n. Refining career goals 


33% 


49% 


15% 


3% 


o. Career opportunity information 


36% 


46% 


18% 


0% 



9. How often are you involved with student organizations? 

Often Sometimes Rarely/Never Don't know/NA 

5% 13% 75% 8% 

10. How satisfied are you with the student organizations? 

Very Somewhat Not at all NA 

5% 43% 10% 43% 



Very Somewhat Not at all 

11% 58% 32% 



12. How would you characterize your friends’ support of your involvement in criminal justice 
classes? 



Extremely 

supportive 



33% 



Quite Somewhat 

supportive supportive 

50% 15% 



Not very 
supportive 



3% 



13. How would you characterize your immediate family’s support of your involvement in criminal 
justice classes? 

Extremely Quite Somewhat Not very 

supportive supportive supportive supportive 

70% 23% 3% 0% 



14. Why are you taking criminal justice classes? (Please respond to each item) 

Secondary 

Primary goal goal Not a goal N/A 
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a. Complete a certificate program 


80% 


13% 


0% 


8% 


b. Obtain an associate degree 


75% 


23% 


3% 


0% 


c. Transfer to a four-year college or university 


28% 


53% 


20% 


0% 


d. Obtain or update job-related skills 


68% 


20% 


10% 


3% 


e. Self-improvement/personal enjoyment 


68% 


28% 


5% 


0% 


f. Change careers 


30% 


13% 


58% 


0% 


15. How are you financing your education? 


( Please respond to each item) 

Major Minor 


Not a 


Not 




Source 


Source 


Source 


Answered 


a. My own income/savings 


45% 


38% 


18% 


0% 


b. Parent or spouse/significant other's 
income/savings 


33% 


18% 


48% 


3% 


c. Employer contributions 


13% 


8% 


75% 


5% 


d. Grants & scholarships 


28% 


15% 


50% 


8% 


e. Student loans 


35% 


15% 


45% 


5% 


f. Public assistance 


10% 


8% 


75% 


8% 


16. What is your overall grade average? 

Do not have a 
GPA at this 

A A- to B+ B B- to C+ C C- or lower school Pass/fail classes only 


35% 33% 25% 8% 


0% 0% 


0% 




0% 


17. My educational experience in the criminal justice department has been: 

Excellent Good 


Fair 


Poor 




65% 


33% 


3% 


0% 


18. Age 


Under 18 18 to 19 20 to 21 22 to 24 25 to 29 


30 to 39 40 to 49 50 to 64 


65+ 


0% 10% 48% 


23% 8% 


13% 0% 


0% 


0% 


19. Sex 


Male 

75% 




Female 

25% 




20. Marital status 


Married 


Single (never 


Divorced 


Widowed 


13% 


married) 

80% 


5% 




3% 



21. Racial grouping (Mark all that apply) 
American Indian or Asian, Asian 

other Native American or 



Native 



Black or 
African 



White, Non- 



Hispanic, 

Latino, 



American Pacific Islander 


Hawaiian 


American 


Hispanic 


Spanish 


8% 0% 


0% 


10% 


83% 


0% 


22. Parents’ education 














Father 


Mother 


Both 


a. Not a high school graduate 




4% 


1% 


5% 


b. High school diploma or GED 




13% 


8% 


28% 


c. Some college, did not complete degree 




3% 


10% 


3% 



Othe 

r 

0% 
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d. Associate’s degree 


3% 


3% 


3% 


e. Bachelor’s degree 


4% 


4% 


3% 


f. Master’s degree/first professional 


1% 


4% 


0% 


g. Doctorate degree 


0% 


0% 


3% 


h. Unknown 


0% 


0% 


0% 



23. Are you the first member of your family to attend college? 

Yes No 

18% 83% 




170 



Appendix O 
Faculty Survey Results 
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Faculty Survey Results 

1. During this school year, how regularly have you had students do the following in your criminal 
justice classes? 

Very Ofte Sometime Neve 





Often 


n 


s 


r 


a. Contribute to class discussions and/or ask questions in class 


62% 


23% 


15% 


0% 


b. Make a presentation in your class 

c. Prepare more than one draft of a paper or assignment before turning in 


0% 


46% 


38% 


15% 


the final work 

d. Integrate concepts or facts from various sources in a paper or project 


0% 


15% 


54% 


31% 


for class 


0% 


62% 


38% 


0% 


e. Fail to complete readings or assignments prior to class 


8% 


33% 


58% 


0% 


f. Prepare projects with other students during class 


15% 


23% 


46% 


15% 


g. Prepare assignments with other students outside of class 


8% 


23% 


54% 


15% 


h. Explain concepts or ideas to other students in preparation for class 

i. As an assignment/project for a class, participate in a community-based 


8% 


31% 


38% 


23% 


project 


0% 


23% 


38% 


38% 


j. Use the internet or instant messaging to prepare an assignment 


25% 


17% 


33% 


25% 


k. Communicate with you via e-mail 


23% 


38% 


38% 


0% 


1. Talk to you about their grades or assignments 


23% 


46% 


31% 


0% 


m. Hold a discussion with you regarding their career plans 

n. Continue classroom discussions or discuss ideas from the readings with 


23% 


31% 


46% 


0% 


you after the class period ended 


8% 


38% 


38% 


15% 


o. Request prompt written or oral feedback regarding their performance 

p. Exceed your own expectations, or work beyond what you believed 


23% 


23% 


46% 


8% 


were their capabilities to meet your standards or expectations 


8% 


31% 


62% 


0% 


q. Interact with you on activities other than coursework 

r. Find them engaging other students, family or co-workers in discussions 


15% 


15% 


54% 


15% 


centered on their readings or classes 


0% 


23% 


62% 


15% 


s. Seriously discuss ideas with students of a different race or ethnicity 

t. Seriously discuss ideas with students holding different religious beliefs. 


8% 


23% 


46% 


23% 


political opinions, or personal values than those held by themselves 


8% 


31% 


38% 


23% 


u. Fail to attend class without an excuse 


8% 


0% 


92% 


0% 
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2. During this school year, what emphasis have you placed on the following? 

Very 

much 

a. Memorization: Repeating facts, ideas, or methods from courses and 


Quite 

bit 


a Som 
e 


Very 

little 


readings in practically the same form in which they were presented 
b. Analysis: breaking down an idea, experience or theory to its basic 


31% 


23% 


46% 


0% 


elements 

c. Perspective: finding new ways to organize and integrate ideas, 


46% 


46% 


8% 


0% 


information, or experiences 

d. Evaluation: assessing the value or soundness of information, arguments. 


23% 


38% 


38% 


0% 


or methods 

e. Adaptation: finding new applications of theories or concepts to practical 


23% 


69% 


8% 


0% 


problems or in new situations 

f. Proficiency: performing a new skill using information students have read 


38% 


46% 


15% 


0% 


or heard 


46% 


31% 


23% 


0% 



3. What amount of reading and writing do you believe students have done during this school year? 



a. Assigned texts, books, manuals, or 


None 


1 to 4 


5 to 10 


11 to 20 


More than 20 


book-length course readings 

b. Books read on their own for personal 


0% 


62% 


23% 


15% 


0% 


enjoyment or academic enrichment 


0% 


62% 


8% 


0% 


0% 


c. Assigned written papers or reports 


0% 


54% 


23% 


23% 


0% 



4. How challenging have you made your examinations during the current school year? 



Extremely 6 


5 4 


3 


2 


Extremely 


difficult 7 

0% 23% 


54% 8% 


8% 


undemanding 

1 

8% 0% 


5. How much emphasis do you place on each of the following? 

Very much 


Quite a bit 


Some 


Very little 


a. Spending substantial amounts of time studying 


15% 


54% 


31% 


0% 


b. Supporting students’ success in the program 


54% 


46% 


0% 


0% 


c. Encouraging students to interact with those of 

diverse economic, social, racial, or ethnic backgrounds 46% 


23% 


23% 


8% 


d. Assistance with students’ non-academic 
responsibilities to your family, employer, etc. 


23% 


38% 


31% 


8% 


e. Support for social interaction and growth 


15% 


46% 


38% 


0% 


f. Making students aware of financial support 
opportunities to help finance their education 


23% 


8% 


62% 


8% 


g. Computer uses in students’ school work 


31% 


31% 


31% 


8% 
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6. During a full seven-day week, how much time, in hours, do you believe your students normally 
spend on each of the following activities? 

More than 







None 


1 to 5 


6 to 10 


11 to 20 


21 to 30 30 


a. Preparing for class 




0% 


92% 


8% 


0% 




0% 0% 


b. Employment for pay 




0% 


0% 


15% 


54% 




31% 0% 


c. Participating in college-sponsored 
activities outside the classroom 

d. Providing care for family members 


0% 


89% 


11% 


0% 




0% 0% 


in their home 




0% 


40% 


30% 


30% 




0% 0% 


e. Travel to and from classes 


0% 


92% 


8% 


0% 




0% 0% 


7. How would you rate your students’ relationships with others in the criminal justice department? 


a. Other Students 


Friendly, 

supportive, 












Unfriendly, 




sense of 












unsupportive, 




camaraderie 












sense of 




7 


6 


5 


4 


3 


2 


hostility 1 




15% 


46% 


31% 


0% 


8% 


0% 


0% 


b. Instructors 


Available, 












Unavailable, 




supportive, 












unsupportive, 




sensitive 7 


6 


5 


4 


3 


2 


insensitive 1 




0% 


46% 


54% 


0% 


0% 


0% 


0% 


c. Office and 

Administrative 

Personnel 


Helpful, 

understanding, 












Unhelpful, 




flexible 












uncaring, rigid 




7 


6 


5 


4 


3 


2 


1 




8% 


15% 


38% 


15% 


8% 


0% 


8% 
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8. Have your students’ skills, and personal development in the following areas been improved by the 
criminal justice department? 





Very much 


Quite a bit 


Some 


Very little 


a. A broad general education 


31% 


38% 


23% 


8% 


b. Job or work-related knowledge and skills 


67% 


33% 


0% 


0% 


c. Clear and effective writing skills 


8% 


38% 


54% 


0% 


d. Clear and effective speaking skills 


23% 


46% 


31% 


0% 


e Critical and analytical thinking 


17% 


67% 


17% 


0% 


f. Numerical problem-solving skills 

g. Information technology and computer 


42% 


33% 


17% 


8% 


skills 


8% 


50% 


33% 


8% 


h. Ability to work effectively with others 


25% 


50% 


25% 


0% 


i. Skills to learn effectively on their own 


15% 


46% 


38% 


0% 


j. Understanding themselves 

k. Understanding people of diverse 


25% 


42% 


33% 


0% 


racial/ethnic backgrounds 
1. Personal code of values and ethics 


25% 


8% 


58% 


8% 


development 


31% 


38% 


31% 


0% 


m. Contributing to community welfare 


27% 


36% 


36% 


0% 


n. Refining career goals 


23% 


62% 


15% 


0% 


o. Career opportunity information 


15% 


54% 


31% 


0% 



9. How often are you involved with student organizations? 

Often Sometimes 



8% 



23% 



Rarely/Never Don’t know/ 
N.A. 

62% 8% 



10. How satisfied are you with the support student organizations provide your students? 

Very Somewhat Not at all N.A. 

8% 83% 0% 8% 



11. How important are student organizations to your students? 

Very Somewhat 

0% 85% 



Not at all N/A 

0% 16% 



12. How would you characterize the support your students receive from their friends of their 
involvement in criminal justice classes? 

Extremely Quite Somewhat Not very 

supportive supportive supportive supportive 

0% 54% 46% 0% 



13. How would you characterize the support your students receive from their families of their 
involvement in criminal justice classes? 

Extremely Quite Somewhat Not very 

supportive supportive supportive supportive 

8% 58% 33% 0% 
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14. Regarding each of the following, how would you characterize your student’s goals for taking 
criminal justice classes? (Please respond to each item ) 

Primary goal Secondary goal Not a goal 



a. Complete a certificate program 

b. Obtain an associate degree 

c. Transfer to a 4-year college or university 

d. Obtain or update job-related skills 

e. Self-improvement/personal enjoyment 

f. Change careers 



54% 


23% 


23% 


69% 


31% 


0% 


25% 


58% 


17% 


54% 


38% 


8% 


23% 


31% 


46% 


15% 


54% 


31% 



15. How do you believe your students are financing their education? (Please respond to each item ) 

Major source Minor source Not a source 

a. Personal income/savings 77% 8% 8% 

b. Parent or spouse/significant other's 

income/savings 23% 46% 15% 

c. Employer contributions 8% 8% 69% 

d. Grants & scholarships 0% 69% 15% 

e. Student loans 62% 15% 8% 

f. Public assistance 15% 46% 23% 



16. How do you believe your students would rate their educational experience in the criminal justice 
department? 

Excellent Good Fair Poor 

46% 54% 0% 0% 
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Appendix P 

Evaluation Panel Members 




Ill 



Members of the Evaluation Panel 

Retired Vice President for Institutional Advancement at a Two-Year Technical College in 
the Midwest 

Qualifications: B.A., English, Cornell College, Mt. Vernon, IA; MEAS, Education, UW- 
Green Bay, WI; Ed.D., Higher Education, Nova Southeastern University. 

Reasons for selection for Evaluation Panel: This member possesses experience as an 
evaluator focusing on assessment and evaluation. This member has been an evaluation 
consultant for a variety of public and private institutions as well as in business and 
educational settings. This member has received accolades for the quality of the member’s 
work products and has also held positions as Executive Director for Institutional 
Advancement, Director of Planning, Research and Development, and Projects and 
Research Services Manager. 

Director of Institutional Effectiveness at a Community College in the Southwest 

Qualifications: B.S., Economics, University Wisconsin-Madison, Madison, WI; Master 
of Arts, Economics, University of Wisconsin-Madison, Madison, WI.; and Ed.D., Higher 
Education, Nova Southeastern University. 

Reasons for selection for Summative Committee: This member possesses 25 years of 
experience in the area of research and evaluation. Past positions held include assessment 
coordinator, research and evaluation specialist, community economic development 
specialist, and economic development planner. This member has published an article in 
the Community College Journal of Research and Practice. This member is currently a 
member of American Evaluation Association and Association for Institutional 
Researchers. 
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Appendix Q 
Evaluation Panel Report 
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Section III 

Evaluation and Gap Analysis 



This section will evaluate the validated criteria of the current student engagement 
practices use by the criminal justice division. The criteria will be evaluated by 
processes and practices identified in the literature review. The researcher has 
identified criteria of student engagement practices at the criminal justice division of 
the college. These practices will be compared in order to create a gap analysis. The 
gap analysis will be completed and viewed by an expert evaluation panel. 
Evaluators, please mark the appropriate number demonstrating the ability of the 
criminal justice division student engagement practices meeting the criteria in 
comparison to best practices as listed in Section II of this document. 



Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4=Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


1. Facilitates student interaction. 


5 


4 


3 


2 


1 



Comments for 1 . 

Identical to Outcome 2, Sec 1 



2. Participate in activities that promote 


5 


4 


3 


2 


1 


engagement. 













Comments for 2. 

Identical to Outcome 3, Sec 1 
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Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


3. Creates an environment where students 

and faculty interact. 


5 


4 


3 


2 


1 


Comments for 3. 

Identical to Outcome 4, Sec 1 


4. Increases critical thinking skills. 


5 


4 


3 


2 


1 


Comments for 4. 

Identical to Outcome 5, Sec 1 


5. Creates an atmosphere for ethnically 

diverse information. 


5 


4 


3 


2 


1 


Comments for 5. 

Identical to Outcome 6, Sec 1 



181 



Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


6. Increases student interaction in class 

activities. 


5 


4 


3 


2 


1 


Comments for 6. 

Identical to Outcome 7, Sec 1 


7. Engagement is affected by socio- 

economic factors. 


5 


4 


3 


2 


1 


Comments for 7. This item does not seem appropriately worded as a “criterion.” It 
seems like a factual statement. The concept is weakly related to Outcome 6 but needs 
restatement, e.g., “Engagement practices address the socio-economic diversity of 
students.” 


8. Importance of engagement practices is 

noted by faculty. 


5 


4 


3 


2 


1 


Comments for 8. 

Somewhat related to Outcomes 4 and 7, may be implied by them, but no statement in 
Section 1 specifically states that faculty note engagement practices 


9. Student engagement activities are well 

crafted. 


5 


4 


3 


2 


1 


Comments for 9. 

Related to Outcomes 2, 3, and 7 in Section 1 
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Criteria to be compared 








QJ 




QJ 






The current student engagement process at the 
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5 


10. Individual meetings occur between 




5 


4 




3 


2 




l 


faculty and students. 



















Comments for 10. 

Although somewhat related to Outcomes 4 and 7, none of those criteria mention 
individual meetings with students. The FVTC criteria are broader in scope than simply 
calling for individual meetings. 



1 1 . Engagement practices include past 
experiences of students. 



5 



4 



3 



2 



1 



Comments for 11. 

Outcomes in Sec 1 contain nothing specific about students’ past experiences. 



12. Collaboration is included in projects that 
involve problem solving. 



5 



4 



3 



2 



1 



Comments for 12. 

# 3 in Sec 1 encompasses teamwork and #5 in Sec 1 addresses problem solving. 



13. Included in courses are situational 
learning activities. 



5 



4 



3 



2 



1 



Comments for 13. 

Outcomes contain nothing specific about situational learning (although we am fairly 
sure it exists in the program) 
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Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


14. Program curriculum is structured to 
form learning communities. 


5 


4 


3 


2 


1 


Comments for 14. 

The outcomes in Section 1 contain nothing specific about learning communities. 


15. Courses contain experiential learning 
components 


5 


4 


3 


2 


1 


Comments for 15. 

The outcomes contain nothing specific about experiential learning. 



Overall comments 

1. The existing criminal justice program outcomes could be amended to encompass 
experiential learning, situational leaning, and learning communities, particularly 
as we believe these actually exist in the criminal justice program. They just aren’t 
specified in the outcomes statements. For instance, the tactical decision making 
lab in the criminal justice program, an application of virtual reality, involves 
students in situational learning. The outcomes may understate what the program is 
actually doing. 

2. In Section 2, item 8 does not seem well-stated as a criterion. Socioeconomic 
diversity would fit in very well as an enhancement to Outcome 7 in Section 1, 
which already addresses ethnic diversity. 
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Executive Summary 

The topic of criminal justice student engagement arose following the release of the 2004 
Community College Survey of Student Engagement results for the college. This 
discussion occurred between the current criminal justice dean and the department chair. 
The discussion included input from faculty about the lackluster performance and limited 
academic involvement of criminal justice students. The study was undertaken as an effort 
to evaluate current engagement activities and how those activities compared with those 
identified in the scholarly literature. 

While there were no preconceptions regarding the state of student engagement practices 
within the criminal justice division, the researcher anticipated the practices could be 
improved and expanded. The Varcoe action research model was employed for this study. 
This model assisted in the identification of current, effectively utilized engagement 
practices. The Varcoe model also served as a guide in gathering information from 
stakeholders, data, and the literature review. 

Formative and summative committees were utilized to validate the criteria, the evaluation 
tool, and the evaluation report. The committees were composed of criminal justice 
educators, practioners, and college administrators. 

The validated evaluation tool was comprised of criteria based upon current criminal 
justice student engagement practices, and validated criteria derived from a review of the 
scholarly literature. The criteria were listed in the evaluation tool to facilitate a gap 
analysis. An expert evaluation panel reviewed the validated evaluation tool and 
completed the gap analysis. The membership of this panel was external to the college. 

The research design incorporated the evaluation tool. The tool aided in identifying gaps 
between current criteria and criteria identified in the literature. The criteria for the 
proposed recommendations were identified through a review of relevant literature and 
surveys of students and faculty members. 
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Key Results 

a. The evaluation panel identified discrepancies with 7 of 16 criteria. The discrepancies 
were between the current division practices and the practices identified through the 
literature review. 

b. Five main gaps were identified as a result of this evaluation. 

c. The evaluation results revealed nine criteria relevant to student engagement within the 
criminal justice division are congruent with practices identified through the literature 
review and criteria identified through the surveys. 

d. Five recommendations were submitted based upon the gap analysis results. 
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Purpose of the Process 

Introduction 

Student engagement is an academic process that has a protracted history. During 
the past three decades, research and writing discussed student engagement as an 
important aspect of quality student learning. This applied dissertation project was 
conducted at a midwestern technical college, hereafter referenced as the college. 

The Wisconsin Technical College System 

The Wisconsin Technical College System, referenced as WTCS, was originally 
established in 1907. In 1917, the focus changed from industrial education to vocational 
education. In 1937, the WTCS expanded, offering college transfer credits to attendees. 
The 1963 Federal Vocational Education Act helped transform the WTCS once again. In 
1965, the State Legislature required the establishment of a statewide vocational, 
technical, and adult education system (Wisconsin Technical College System History, 
2006). 

The WTCS, of which the college remains a member, is comprised of 16 technical 
colleges sited throughout the state. Each institution serves a geographically defined area 
referred to as a district. Each technical college is directed to serve the needs and 
requirements of its individual labor markets. 

Mission and Vision 

In order to address the academic and learning needs of students and the functions 

the college system provides to meet those needs, the legislature assigned the WTCS the 

following mission statement in the enabling legislation (Wisconsin Statutes, 2005-06): 

The legislature finds it in the public interest to provide a system of technical 
colleges which enables eligible persons to acquire the occupational skills training 
necessary for full participation in the work force; which stresses job training and 
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retraining; which recognizes the rapidly changing educational needs of residents 
to keep current with the demands of the work place and through its course 
offerings and programs facilitates educational options for residents; which fosters 
economic development; which provides education through associate degree 
programs and other programs below the baccalaureate level; which functions 
cooperatively with other educational institutions and other governmental bodies; 
and which provides services to all members of the public. (§38.001) 

In addition to the mission, the Wisconsin Technical College System Vision 

(2006), further addresses the provision of meeting the academic and learning needs of 

students in the following vision statement: 

The Wisconsin Technical College System is the premier provider of technical 
education. We develop individuals who apply knowledge and skills to enhance 
quality of life and boost economic vitality. We are committed to extending 
learning beyond the classroom and throughout life. 

To meet each student’s educational needs, we: 

Deliver high quality instruction and services that are responsive, flexible and 
accessible. 

Join talent and technology to make learning generously available and 
imaginatively delivered. 

Commit to high standards and accountability. 

Create strategic alliances that expand students’ learning opportunities. 
Respect each other’s dignity, embrace diversity and offer opportunities for 
growth. (*![ 1) 

The Midwestern Technical College 

In addition to the WTCS vision and mission, each individual college is mandated 
by WTCS to maintain a mission and vision statement and that mirrors the mission and 
vision statement of the WTCS. The mission statement of the college stresses the 
importance of aiding individuals as they strive to reach their potential. The college 
provides cost-effective education and training and prepares students for successfully 
meeting their objectives for employment, continuing higher education, and personal 
enrichment. This statement mirrors the legislative intent and direction to prepare district 
residents for participation in the state’s workforce. 
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The vision of the college, as published on the college web site states: 

The Strategic Planning Process has led us to a shared vision to fulfill our mission. 
Four strategic directions focus on assuring excellence in all College endeavors, 
maximizing the collective value of those endeavors in the eyes of the College’s 
constituency, reaching more people with programs and services, and 
strengthening the lifelong connection between the College and its students and 
graduates. 

Protect and enhance the performance reputation of the College by seeking 
continuous improvement in all areas. 

Strengthen the College’s value to its constituents. 

Reach more people with programs and services. 

Strengthen lifelong ties to our students and graduates. ( ( J[ 1) 

The criminal justice division is the largest academic area of the college. The 

division offers associate degrees in criminal justice and in security and loss prevention. 

The division offers three technical certificates in the areas of cyber crime investigations, 

criminal justice leadership, and private investigation. The division also operates three 

entry-level academies, leadership and management courses, and a plethora of specialized 

law enforcement trainings. The division has a complement of six full-time and 42 adjunct 

faculty. The division has assimilated the mission and vision statements of the college. 

Nature of the Problem 

The criminal justice division enjoys an excellent reputation throughout the state. 
The college 2006 Graduate Employment Research Report demonstrates the strength of 
this reputation. The report shows a 90% employment rate six months after graduation. 
The criminal justice division has a reputation that masks the problem studied. The 
problem is that criminal justice students fell below the 2004 Community College Survey 
of Student Engagement mean in four of the five engagement benchmarks. 

The college participated in the 2004 Community College Survey of Student 
Engagement, referenced as CCSSE. The survey is administered through the Community 
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College Leadership Program of the University of Texas. The survey was administered to 
532 full-and 228 part-time students. The criminal justice division was disproportionately 
represented in the sample. Of the 760 student participating in the survey 283, or 37%, 
were criminal justice students 

The study reported on five benchmarks of student engagement. These benchmarks 
were (a) active and collaborative learning, (b) student effort, (c) academic challenge, (d) 
student-faculty interaction, and (e) support for learners (Community College Leadership 
Program, 2004). Student engagement refers to the amount of effort students invest in 
their learning. Active and collaborative learning relates to how actively involved students 
are in their learning. The academic challenge benchmark inquires into the complexity of 
the cognitive work and course evaluation standards. Academic challenge examines 
whether students are afforded the opportunity to solve problems with peers and to 
challenge course content. How the faculty engages students was also surveyed using the 
student-faculty interaction benchmark, which examined the involvement of instructors 
with students. 

The final benchmark focused on support for learners. In the report Engagement by 
Design: 2004 Findings (Community College Leadership Program, 2004), it was noted 
such services “may affect learning and retention” (p. 12). 

The CCSSE established that the college was above the mean, 51.8%, for the 
active and collaborative learning benchmark. The director of institutional research 
reported the college was below the mean in the remaining four benchmarks, as indicated 
in an unpublished institutional report of the 2004 CCSSE. The result indicated that the 
below mean benchmarks were statistically significant. The mean for the benchmarks was 
determined always to be 50% (Community College Leadership Program, 2004). 
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Background and Significance of the Problem 

The issue of student preparation for college is important not only for four- year 
institutions, but especially so for two-year colleges. Astin (1985) commented that in 
“virtually every longitudinal study of student development - is that the chances of 
dropping out are substantially greater at a two-year college than at a four-year college” 

(p. 146). 

The college is accredited through the North Central Association of Colleges and 

Schools. The college participates in the Academic Quality Improvement Program, 

referenced as AQIP, for reaccreditation. The college was admitted to AQIP on October 3, 

2001. The next reaccreditation process will be conducted during the 2008-2009 academic 

year (Higher Learning Commission, 2002). The North Central Association of Colleges 

and Schools, Higher Learning Commission, in describing the AQIP process wrote: 

... the Academic Quality Improvement Program infuses the principles and 
benefits of continuous improvement into the culture of colleges and universities 
by providing an alternative process through which an already-accredited 
institution can maintain its accreditation from the Higher Learning Commission. 
With AQIP, an institution demonstrates it meets accreditation standards and 
expectations through sequences of events that align with those ongoing activities 
that characterize organizations striving to improve their performance, (p. 1) 

The AQIP process examines several areas of the college. One of these areas is 

how the college helps students learn. Within the scope of helping students to learn, AQIP 

inquires how the college aligns co-curricular activities with course curriculum objectives. 

Purpose of the Project 

The purpose of this study was to evaluate the engagement process of criminal 
justice majors at the college. Based on the results of the evaluation of current student 
engagement practices, this recommendation is being forwarded to the dean of the 
criminal justice division and the executive vice president/chief academic officer of the 
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college. The purpose of this recommendation is to facilitate opportunities for faculty to 
assist students discover ways to become more involved in the classroom and engage their 
academic assignments with vigor. 

Research Questions 

To conduct the student engagement evaluation, five research questions were 
proposed. The purpose of the questions was to provide direction for the study to evaluate 
the student engagement process at the college. 

1. What current engagement activities are available for criminal justice students at 
the college? The rationale for this question was to identify existing practices and to 
establish a foundation for the identification of a model of student engagement activities. 

2. What models are identified in the literature of expected outcomes for best 
practices of successful student engagement processes and programs? The rationale for 
this question was to determine potential engagement practices to determine their 
appropriateness to be included in the recommendation to the dean of the criminal justice 
division and the executive vice president/chief academic officer 

3. What are the perceptions of students and instructors regarding student 
engagement at the college? The rationale for this question was to obtain feedback from 
students and faculty regarding their opinions of student engagement practices currently 
utilized by the division. 

4. What valid components are necessary to formulate an evaluation tool that will 
assess the worth of the engagement practices currently offered to criminal justice 
students? The rationale for this question was to construct a valid evaluation tool. 

5. What would be included in a valid recommendation to revise, implement and 
evaluate an improved academic engagement process? The rationale for this question was 
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to provide a viable recommendation regarding student engagement to the dean of the 
criminal justice division and the executive vice president/chief academic officer. 

Methodology 

Methodology 

The evaluation research methodology was utilized as an action research study to 
evaluate student engagement at a midwestern technical college criminal justice program. 
Charles and Mertler (2002) wrote, “Evaluation research is done to determine the relative 
merits of various products and approaches used in education” (p. 3 1 1). The study 
attempted to identify current engagement practices and any gap between existing 
practices and the expected criteria of best scenarios and to identify concepts that may 
support an improved engagement process. Airasian and Gay (2003) supplied 
reinforcement for using the evaluative method as they described the purpose of 
evaluation research. The description provided stated “Evaluation research is concerned 
with making decisions about the quality, effectiveness, merit, or value of educational 
programs, products, or practice” (p. 7). 

The Tyler objectives-oriented approach is a research method designed to evaluate 
whether program objectives are being met (Fitzpatrick, Sanders, & Worthen, 2004). The 
focus of this study was the engagement practices that were utilized to enable students to 
meet course and program objectives. Varcoe (2005) described an evaluation study as 
follows: 

An evaluation study is a carefully implemented set of procedures used to assess 
the value, merit or worth of a program, practice, activity, product, process, etc. 
when compared with pre-established criteria. The comparison of the observed 
(assessment information) to the criteria-called ‘gap analysis’ -produces the 
findings. These findings are the basis for recommendations and/or decisions, (p. 
8 ) 
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The data collection process utilized two survey tools administered to participants 
for self-reporting of information. One assumption associated with self-reporting is that 
the participants will honestly report. Charles and Mertler (2002) made that assumption 
based upon the premise that the survey will prompt a process that will “elicit and probe 
responses from participants and informants” (p.162). 

Pike and Kuh (2005) addressed the issue of inaccurate self-reporting. They wrote 
that a well-written survey will negate inaccurate responses and identified five conditions 
that will enhance survey accuracy. First, ask questions to which participants know, or are 
reasonably expected to know, the answer. Second, phrase the questions in a manner that 
is clear and explicit. Third, refer to recent occurrences. Fourth, elicit an insightful 
response from the participants. Fifth, respect the participants in order to elicit honest 
answers. 

Procedures 

Formative and summative committees were formed to assist the researcher with 
developing criteria and the evaluation tool. Fifteen criteria were developed for the 
evaluation tool. Criteria were developed using the results of two surveys and a 
comprehensive literature review. One survey, the student survey, was administered to 40 
criminal justice students. The second survey, the faculty survey, was administered to five 
full-time and ten adjunct faculty members. All participants were provided a participation 
letter/informed consent form to read and review. Survey results were collected and 
secured in a locked file cabinet to assure participant confidentiality. The collected data 
aided in identifying criteria and contributed to the development of the evaluation tool. 
The tool was considered valid after its review by the formative and summative 



committees. 
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The researcher forwarded the validated evaluation tool via e-mail to the expert 
evaluation panel, which consisted of two members external to the college. The evaluation 
was completed, and the members provided their results to the researcher. The researcher 
examined the data and prepared an evaluation report. The evaluation report was provided 
to the formative and summative committee members for review and validation. 

The researcher anticipates that the results of this evaluation study will facilitate 
implementation of recommendations for improving student engagement practices at the 
criminal justice division. Literature suggests that students who participate in college 
activities, both academic and co-curricular, are successful. The recommendations of this 
study will provide opportunities to improve student engagement at the criminal justice 
division. 

Assumptions 

Several assumptions were made in the course of this study. First, the researcher 
assumed the reviewed literature exhausted the available information on the topic of 
engagement, and the information contained in the literature was accurate. Second, the 
researcher assumed the members of the formative, summative, and expert committees 
utilized their knowledge to the best of their abilities to provide input and validation to the 
study. Third, the researcher assumed that the students participating in the survey study 
were representative of the general criminal justice division student population. Fourth, 
the researcher assumed the college personnel being surveyed were forthright regarding 
their responses to the survey questions. Fifth, the researcher assumed the surveys were 
reliable and valid as stated by feedback from the formative and summative committees 
and the local institutional review process. 



Limitations 
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In discussing limitations, Charles & Mertler, (2000) wrote that “Limitations refer 
to conditions outside the investigator’s control that affect data collection” (p. 71). Several 
limitations for this study were identified. First, varying degrees of knowledge and 
expertise among the formative and summative committee members existed. Second, the 
motivations, commitments, and expectations of the formative and summative committee 
members impacted the direction of the research study. Third, since the study focused on 
one department at a two-year technical college, the results may not be generalized to 
other educational institutions or other divisions or departments within the college. Fourth, 
not all participants would respond to the survey. Fifth, changes based upon certification 
requirements for law enforcement officers are also being undertaken. These changes are 
ongoing and being instituted by a non-educationally focused agency. Such changes will 
materially affect course structure. As these changes occur, the criminal justice program 
will adjust to incorporate them, and the relevance of this evaluation study may diminish 
as a result of the adjustments. 

Implications for the Improvement of Educational Practice 

Three outcomes of this study contribute to improving educational practice. First, 
the study provides the college with additional resources to facilitate the learning of 
students enrolled in the criminal justice program. Second, this research project 
contributes to the body of research focusing on student engagement in two-year colleges. 
Third, the results of this study identify any gap in the use of current student engagement 
practices presently utilized by criminal justice faculty. 

Results 

Gap Analysis Results 

The evaluation indicates that opportunities for student engagement currently exist 
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in the criminal justice curriculum. Students and faculty report engagement activities are 
regularly utilized. Students and faculty agree that the criminal justice division is 
supportive of students, both on academic and personal levels. The evaluation panel 
indicates that the incorporation of student engagement activities by the criminal justice 
division was deficient in seven areas. These areas were identified through a gap analysis. 
Gaps identify areas for improvement. Recommendations for resolving the seven gaps are 
a result of this evaluation. Three of the analyzed criteria are interrelated. Discussion of 
these criteria was combined. In total, five gaps were identified, and recommendations to 
address these gaps prepared. 

The first identified gap related to student-faculty individual meetings. The college 
identifies the importance of student-faculty interaction, as evidenced through the 
scheduling of instructor office hours. While office hours are identified as necessary, and 
time is allotted for this purpose, a gap continues to exist. Office and advising hours are 
scheduled by faculty based upon faculty preference. The scheduling of class hours is not 
a consideration in determining office hours, and this may result in the existence of the 
gap. A second cause for this gap may be related to the manner in which the faculty 
advising process is explained to students during the new student orientation. 

The second identified gap relates to capitalizing upon students’ life experiences. 
Most students in the program meet the definition of a traditional student. The definition 
identifies this student as being between the ages of 18 and 22 and being enrolled in 
course work full-time. The presence of this gap may exist due to limited student life 
experiences relative to criminal justice activities. Evaluating the scope of student life 
experiences in a class is difficult. Students may be unwilling to discuss past personal 
involvement with law enforcement authorities within a classroom environment. 
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The curriculum as presently constructed provides a limited number of 
opportunities to engage students in situational learning activities. Courses currently 
include scenario-based activities and assessments, narrowly focused on state law 
enforcement officer certification requirements, not on providing the students with 
engagement opportunities. The narrow focus limits the breadth of possibilities for 
situational learning. 

A third gap revealed in the evaluation is that the curriculum is not structured to 
form learning communities. The program currently allows students to “self-select” their 
course schedules. Students attend various courses together, but this is due to 
happenstance or student preference rather than by curriculum design. No coordination 
exists between criminal justice courses and applicable required general education 
courses. 

A fourth gap is the importance faculty place on student engagement. The degree 
of importance a particular faculty member places on engagement-related activities is 
difficult to measure. The panel indicated that faculty do not place importance on student 
engagement activities. 

The fifth gap identified is whether the diversity of students’ socioeconomic 
conditions is addressed by the division’s student engagement practices. The student 
survey and the literature review both support the existence of this gap. Many students 
who attend two-year institutions maintain employment during the course of their studies. 
The student survey indicates that 55% of the participants work more than 30 hours per 
week. The survey also identified that students receive little, if any, monetary assistance 
from their parents or tuition reimbursement from their employers. 



Recommendations 
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Five recommendations are presented based upon the results of this study. The 
commitment of the criminal justice division and the college to provide quality education 
indicates that these recommendations will be thoughtfully reviewed and implemented. 

First, the criminal justice division needs to explore and implement practices 
designed to build learning communities. Learning communities will enhance student 
engagement and learning with linked or bundled courses. The law enforcement 
profession requires practitioners to be proficient at many skills. In addition, professional 
law enforcement practioners are expected to possess diversified knowledge covering 
several interrelated disciplines. Linking social science courses with criminal justice 
courses will increase engagement opportunities and facilitate learning in both disciplines. 
Consideration should be directed at linking the introduction to criminal justice course 
with the contemporary American society course. The juvenile procedures course may be 
linked with either introductory psychology or abnormal psychology, with discussions 
between criminal justice and social science faculty to coordinate curriculum and 
complement the instructional goals of each course taking place on a regular basis. 

Bundled courses may be designed to create a student cohort. Through the creation 
of the cohort, advisors direct students to enroll in specific complementary courses. As the 
students interact across different disciplines, the opportunities for engagement increase, 
and learning is facilitated. 

Second, criminal justice faculty need to examine the extent that situational and 
experiential instruction methods are utilized, with a focus on the variety of situations and 
experiences beneficial to the instructional process. The faculty will determine the design 
and placement of student engagement activities that will most benefit student learning. 
The implementation of this recommendation will prevent duplication of activities. 
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Recognition of the impact socioeconomic diversity has on law enforcement 
expands the pool of potential experiential and situational learning opportunities. The 
recognition of this diversity allows faculty to incorporate instructional techniques that 
embrace a wide range student experiences. The college recognizes that many of the 
division students are not experienced in diversity-related issues or situations. By 
encouraging faculty participation in the development of experiential learning 
opportunities, faculty are empowered to accept ownership of the revised curriculum. 

Third, student-faculty communication needs to improve. Improving the new 
student orientation presentation may be helpful. Stressing the importance of student- 
faculty communication during the orientation is a first step toward improvement. The 
orientation provides the opportunity to advise students what they may reasonably expect 
from faculty outside of scheduled instructional hours. 

Fourth, reexamination of the methodology used to compute and schedule office 
and advising hours warrants improvement. Currently, the number of assigned office 
hours is based upon the current number of a faculty member’s assigned instructional 
hours. The number of office hours does not factor in the number of students enrolled in a 
course. No criteria currently exists to determine the number of hours a faculty advisor 
may schedule. 

Fifth, the researcher recommends improving advising services for criminal justice 
students. The current process allows students to “self-advise,” a process that does not 
encourage student interaction with faculty. Assigning faculty advisors to specific students 
is one action that will encourage the development of a relationship between faculty and 
students. The researcher recommends instituting a requirement that a faculty advisor meet 
with a student a minimum of once per term, a requirement that may also increase faculty- 
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student communication and engagement. A registration hold is one option to consider in 
facilitating the advising session. 

Conclusion 

An examination of student engagement practices within the criminal justice 
division was the focus of this evaluation study. The study examined data derived from 
student and faculty surveys as well as a comprehensive literature review. 

The study revealed that the criminal justice division currently uses some student 
engagement practices. However, the evaluation identified several gaps between the ideal 
student engagement practices identified in the literature and those practices currently in 
use. The identified gaps may to be resolvable by administrative action. The 
recommendations made in this report may be a guideline for implementation of these 
changes. 
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Evaluation Report 



Section III 

Evaluation and Gap Analysis 



This section will evaluate the validated criteria of the current student engagement 
practices use by the criminal justice division. The criteria will be evaluated by 
processes and practices identified in the literature review. The researcher has 
identified criteria of student engagement practices at the criminal justice division of 
the college. These practices will be compared in order to create a gap analysis. The 
gap analysis will be completed and viewed by an expert evaluation panel. 
Evaluators, please mark the appropriate number demonstrating the ability of the 
criminal justice division student engagement practices meeting the criteria in 
comparison to best practices as listed in Section II of this document. 



Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4=Agree 


3=Undecided or 
Neutral 


2=Disagree 


1- Strongly 
Disagree 


16. Facilitates student interaction. 


5 


4 


3 


2 


1 



Comments for 1. 

Identical to Outcome 2, Sec 1 



17. Participate in activities that promote 


5 


4 


3 


2 


1 


engagement. 













Comments for 2. 

Identical to Outcome 3, Sec 1 
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Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


18. Creates an environment where students 
and faculty interact. 


5 


4 


3 


2 


1 


Comments for 3. 

Identical to Outcome 4, Sec 1 


19. Increases critical thinking skills. 


5 


4 


3 


2 


1 


Comments for 4. 

Identical to Outcome 5, Sec 1 


20. Creates an atmosphere for ethnically 
diverse information. 


5 


4 


3 


2 


1 


Comments for 5. 

Identical to Outcome 6, Sec 1 
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Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


21. Increases student interaction in class 
activities. 


5 


4 


3 


2 


1 


Comments for 6. 

Identical to Outcome 7, Sec 1 


22. Engagement is affected by socio- 
economic factors. 


5 


4 


3 


2 


1 


Comments for 7. This item does not seem appropriately worded as a “criterion.” It 
seems like a factual statement. The concept is weakly related to Outcome 6 but needs 
restatement, e.g., “Engagement practices address the socio-economic diversity of 
students.” 


23. Importance of engagement practices is 
noted by faculty. 


5 


4 


3 


2 


1 


Comments for 8. 

Somewhat related to Outcomes 4 and 7, may be implied by them, but no statement in 
Section 1 specifically states that faculty note engagement practices 


24. Student engagement activities are well 
crafted. 


5 


4 


3 


2 


1 


Comments for 9. 

Related to Outcomes 2, 3, and 7 in Section 1 
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Criteria to be compared 
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25. Individual meetings occur between 




5 


4 




3 


2 




l 


faculty and students. 



















Comments for 10. 

Although somewhat related to Outcomes 4 and 7, none of those criteria mention 
individual meetings with students. The FVTC criteria are broader in scope than simply 
calling for individual meetings. 



26. Engagement practices include past 
experiences of students. 



5 



4 



3 



2 



1 



Comments for 11. 

Outcomes in Sec 1 contain nothing specific about students’ past experiences. 



27. Collaboration is included in projects that 
involve problem solving. 



5 



4 



3 



2 



1 



Comments for 12. 

# 3 in Sec 1 encompasses teamwork and #5 in Sec 1 addresses problem solving. 



28. Included in courses are situational 
learning activities. 



5 



4 



3 



2 



1 



Comments for 13. 

Outcomes contain nothing specific about situational learning (although we am fairly 
sure it exists in the program) 
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Criteria to be compared 

The current student engagement process at the 
criminal justice division: 


5=Strongly 

Agree 


4= Agree 


3=Undecided or 
Neutral 


2=Disagree 


1-Strongly 

Disagree 


29. Program curriculum is structured to 
form learning communities. 


5 


4 


3 


2 


1 


Comments for 14. 

The outcomes in Section 1 contain nothing specific about learning communities. 


30. Courses contain experiential learning 
components 


5 


4 


3 


2 


1 


Comments for 15. 

The outcomes contain nothing specific about experiential learning. 



Overall comments 

3. The existing criminal justice program outcomes could be amended to encompass 
experiential learning, situational leaning, and learning communities, particularly 
as we believe these actually exist in the criminal justice program. They just aren’t 
specified in the outcomes statements. For instance, the tactical decision making 
lab in the criminal justice program, an application of virtual reality, involves 
students in situational learning. The outcomes may understate what the program is 
actually doing. 

4. In Section 2, item 8 does not seem well-stated as a criterion. Socioeconomic 
diversity would fit in very well as an enhancement to Outcome 7 in Section 1, 
which already addresses ethnic diversity. 
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Appendix S 
Implementation Plan 
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Implementation Plan 

The plan will adhere to college protocols for the revision of curricula. The plan 
will follow the protocol of the college, utilizing the following procedures: 

(a) Plan for Revision: 

a. The criminal justice dean, in consultation with the executive vice 
president/chief academic officer, will determine the recommendations 
appropriate for implementation. 

b. The dean will appoint a criminal justice faculty member to assist in the 
development and implementation of tasks related to implementation. 

c. Criminal justice faculty will meet to review the recommendations, address 
concerns, and develop an implementation plan that assigns specific duties 
to faculty members. 

(b) Plan for Implementation: 

a. Complete implementation of the revision plan will occur within six 
months. 

b. Use monthly staff meetings of criminal justice faculty to discuss progress 
and concerns of the implementation plan. 

c. Faculty will work cooperatively to implement the recommendations 
assigned by the dean. 

d. Faculty will prepare six-month progress reports for the dean. 

e. The executive cabinet will receive a status report regarding the 
implementation plan at the discretion of the executive vice president/chief 
academic officer of the college. 

(c) Plan for Future Evaluation: 

a. A follow-up evaluation will be conducted by the college research and 
development department 18 months after the recommended changes are 
effected. 

b. Internet technology will be the method of choice for the follow-up survey. 

c. Students admitted to the criminal justice program for one academic year 
following plan implementation are eligible for participation. 
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d. Results of the survey will be compared with this study. 

e. If indicated, the implemented recommendations will be revised. 

f. The survey will be re- administered two years following plan 
implementation. 




